1 2 Previous Next 18 Replies Latest reply on Jun 18, 2009 8:03 PM by 807588

    How to read the nth line from a specific file in a better way?

    807588
      Hi,

      I need to read the nth line from a specific file. I am using BufferedReader to read the file line-by-line.

      The logic is as followed:
      1: Find the total number of lines in the file
      2: Generate a random number ranging between 1 and the total no. of lines
      3. Declare a counter for line number
      4. For loop to read the file line-by-line
         {
      5.      Increase the counter
      6.      Check if the random number == counter
              {
      7.           read this line
      8.           exit from the loop
           }
      9.      Increase the counter
          }
      I am calculating the total number of lines (step 1) using the same logic of BufferedReader.

      The code is working as expected. But as I need to keep displaying that random line in an infinite while loop and the file contains ~ 4000 lines, it's creating performance issues.

      Is there any better way to read a particular line of a file without reading the preceding lines?

      Thanks in advance...

      Regards,
      Ujjal
        • 1. Re: How to read the nth line from a specific file in a better way?
          791266
          Is there any better way to read a particular line of a file without reading the preceding lines?
          No


          What do you need to do this? 4000 lines doesn't sound much. You could place all of them in e.g. an ArrayList.

          Kaj
          • 2. Re: How to read the nth line from a specific file in a better way?
            807588
            Hi Kaj,

            Thanks a lot for your reply.

            What I have understood from your reply is:
            At the beginning of the code, read the file once and put each line in an ArrayList<String>. And then get the nth element from that ArrayList.

            Is my understanding correct?

            Regards,
            Ujjal
            • 3. Re: How to read the nth line from a specific file in a better way?
              791266
              Is my understanding correct?
              Correct

              But why do you want to do this?
              • 4. Re: How to read the nth line from a specific file in a better way?
                807588
                Rather than deciding on a random line number in advance you could take a random decision after reading each line. Read the first line and store that. Then read the second line and chose that, or the one you already have on a 50-50 basis. Then read the next line and decide whether to chose that one, or the choice already made. Obviously the more lines you have read the less the chance that the most recent line will be the chosen one. You'd still have to read all the lines, but only once and without storing the whole file in memory.
                • 5. Re: How to read the nth line from a specific file in a better way?
                  807588
                  Well...

                  I need to develop an application for displaying help messages to the users. I have to display these random messages in a JWindow, which will be placed at the top of the screen. The display should be continuous and random. And the sleep period between each message is adjustable by the user (between 1-30 sec). Also, the number of messages is ~2000 now. But that could get doubled/tripled in future. The program has to read a .lst file where all the messages are kept.

                  Now with about 4000 lines, if a user sets the period as 1 sec, the program is consuming lots of memory. So I was looking for alternate/more efficient method to read the lines.
                  • 6. Re: How to read the nth line from a specific file in a better way?
                    800456
                    If you have control of the format of your .lst file, you could make all the lines the same length (padded with whitespace) and then use RandomAccessFile to seek to the a random line by multiplying the random number by the line length.

                    A slight performance gain for the sacrifice of a bigger file.

                    Edited by: andrew_malcolm on Jun 15, 2009 2:18 PM
                    • 7. Re: How to read the nth line from a specific file in a better way?
                      JoachimSauer
                      Ujjal wrote:
                      Now with about 4000 lines, if a user sets the period as 1 sec, the program is consuming lots of memory. So I was looking for alternate/more efficient method to read the lines.
                      If the mesages are short enough to be read in 1 sec and you've got about 4000 of them, then you're probably not using a lot of memory in any meaningful measure (unless you're talking embedded systems).

                      Let's say 100 characters per message, 2 bytes per character, 4000 messages is 100 * 2 * 4000 = 800000 bytes is roughly 781kb.

                      So you're thinking about optimizing stuff that takes less than a megabyte. On a modern computer you won't ever notice this. Actually reading the data from the hard disk every second will be a lot more noticeable for the user than using that amount of memory.
                      • 8. Re: How to read the nth line from a specific file in a better way?
                        807588
                        If you read the whole file into, say, an ArrayList the amount of memory doesn't depend on how frequently it gets accessed (and it's not a huge amount of memory by current standards anyway). My suggestion assumed that the cost of counting the lines was included, but if you keep using the same file it's not a factor.

                        You could get fancy and index the first byte offset of each line, then read the line on a skip-sequential basis, perhaps mapping the file to a ByteBuffer (this does not take up "real" memory).
                        • 9. Re: How to read the nth line from a specific file in a better way?
                          807588
                          Ujjal wrote:
                          Well...

                          I need to develop an application for displaying help messages to the users.
                          I'm not sure how helpful you are to your users if you are displaying random help messages. Are they suppose to wait until a message applicable to their problem come up? This is very strange.
                          Also, the number of messages is ~2000 now. But that could get doubled/tripled in future.
                          You know, if the user was no knowledge about the messages in the file, why don't you just display them one line at a time starting from the beginning. As as far as they are concerned, the displayed messages don't have any sort of order to them.
                          • 10. Re: How to read the nth line from a specific file in a better way?
                            807588
                            Hi,

                            It's not a help message exactly. It's some sort of tips on an application they are working on; similar to what we get as 'Tip of the Day' when we open, say, Texpad. But it has to be continuous throughout the day. This to make the users familiar with the system they are using. :)

                            And it has to be random. Else the whole concept would be a little monotonous/repetitive as users point of view.. :(
                            • 11. Re: How to read the nth line from a specific file in a better way?
                              807588
                              I agree with JoachimSauer. Read all the lines in and keep then in a list. This is the simplest thing to do. Then profile your app.
                              • 12. Re: How to read the nth line from a specific file in a better way?
                                807588
                                Ujjal wrote:
                                Hi,

                                It's not a help message exactly. It's some sort of tips on an application they are working on; similar to what we get as 'Tip of the Day' when we open, say, Texpad. But it has to be continuous throughout the day. This to make the users familiar with the system they are using. :)

                                And it has to be random. Else the whole concept would be a little monotonous/repetitive as users point of view.. :(
                                I see. What others have suggested would work: read in the whole file into an array of List<String> and randomly index a message to display.

                                But realistically, how many messages can a user read? 2000 messages at say 3 seconds for each read is 100 minutes of reading one right after the other. Pretty crazy. So, an argument can be made that not all messages need to be read in if the number of messages is very large.

                                I would suggest filling an array or List<String> of about 2000 messages to randomly display. That should be distracting enough in most cases. To fill this collection of messages I suggest the following:

                                1) randomly pick a number between 1 and the File.length() of the file
                                2) read in each line counting bytes read so far
                                3) compare the byte count with the random number; if equal or greater than, include the last line read for display.
                                4) goto 1) until you've got 2000 messages.

                                Note: may include duplicate messages.

                                Also, you could do this.

                                1) randomly pick a number between 1 and a max of say 64.
                                2) read in each line counting bytes read so far
                                3) compare the byte count with the random number; if equal or greater than, include the last line read for display.
                                4) if need more messages and byte count is less than File.length() - max goto 1)
                                5) if need more messages and byte count is greater than File.length() - max then reset read to beginning of file.
                                • 13. Re: How to read the nth line from a specific file in a better way?
                                  YoungWinston
                                  Ujjal wrote:
                                  The code is working as expected. But as I need to keep displaying that random line in an infinite while loop and the file contains ~ 4000 lines, it's creating performance issues.
                                  ...
                                  As everyone else has said, I don't think there's much way around it; and, once done, it will be blindingly fast.

                                  If you're worried about the space consideration of everyone having their own copy of the file, you might want to think about some sort of shared daemon thread, callable by any user on the system. I'm not familiar with the syntax of setting up or calling such a beast though; perhaps others have tried it. I'd be interested to know myself.

                                  Winston
                                  • 14. Re: How to read the nth line from a specific file in a better way?
                                    807588
                                    YoungWinston wrote:
                                    Ujjal wrote:
                                    The code is working as expected. But as I need to keep displaying that random line in an infinite while loop and the file contains ~ 4000 lines, it's creating performance issues.
                                    ...
                                    As everyone else has said, I don't think there's much way around it; and, once done, it will be blindingly fast.

                                    If you're worried about the space consideration of everyone having their own copy of the file, you might want to think about some sort of shared daemon thread, callable by any user on the system. I'm not familiar with the syntax of setting up or calling such a beast though; perhaps others have tried it. I'd be interested to know myself.

                                    Winston
                                    That would make sense only as a separate process, then you would have the network overhead. Just read the durn file and see if that's fast enough. No sense getting tricky right out of the box.
                                    1 2 Previous Next