5 Replies Latest reply: Jan 4, 2012 5:00 PM by 908528 RSS

    Maximum Size of Scanner String

    908528
      Hello,

      I'm using scanner to read in the contents of a text file to a String, but the file is not being read in compeletely, and the length of the resulting string is 1024. The flie has 4160 characters in it.
      Scanner rScanner = new Scanner(rStream);
      rScanner.useDelimiter("\\z");
      String rCommands = rScanner.next();
      I'm using Java 1.7.0_01. Is this limit documented anywhere ? I'd like to avoid looping and concatenating in my code.

      Thanks,
      Dario.

      --------------------------------------
      Dario Strbenac
      Research Assistant
      Cancer Epigenetics
      Garvan Institute of Medical Research
      Darlinghurst NSW 2010
      Australia
        • 1. Re: Maximum Size of Scanner String
          sabre150
          That is not really an appropriate way to read the whole of a file. My approach that does not have any explicit looping is -

          1) Create a byte array of the length of your file. The length can be obtained from the File object (one line).
          2) Open the File with a FileInputStream and wrap the FileInputStream in a DataInputStream (one line).
          3) Use DataInputStream.readFully(byte[] ) to completely read the file then close the DataInputStream (two lines).
          4) Convert the byte array to a String using String fileContent = new String(byte[], characterEncoding) (one line).

          Total of 5 lines of code, 6 if you have to create the File object.

          P.S. It is rarely right to read the whole of a file into memory since it does not scale well.
          • 2. Re: Maximum Size of Scanner String
            908528
            It's a nice way to do it for files in the file system. However, rStream is acutally reading a text file in from inside the JAR file.
            InputStream rStream = getClass().getResourceAsStream("/R/accessory.R");
            Since there's no File object involved here, I'd have to do it in a loop anyway, right ? Actually, come to think of it, I can just hard-code the size of the byte array, since the text file contains code from another programming language that likely won't be changing in the future.
            • 3. Re: Maximum Size of Scanner String
              DrClap
              No, you could still wrap that InputStream in a DataInputStream and continue with the rest of sabre150's procedure.
              • 4. Re: Maximum Size of Scanner String
                sabre150
                user3454292 wrote:
                It's a nice way to do it for files in the file system. However, rStream is acutally reading a text file in from inside the JAR file.
                So it's a 'resource' and not a 'file' and one cannot get the length of a resource without reading it.

                >
                InputStream rStream = getClass().getResourceAsStream("/R/accessory.R");
                Since there's no File object involved here, I'd have to do it in a loop anyway, right ?
                Yes - but that is almost trivial. Just loop reading N bytes at a time from the resource and write them to a ByteArrayOutputStream then convert the bytes to a String. Alternatively, loop reading the resource N characters at a time and write to a StringWriter.
                Actually, come to think of it, I can just hard-code the size of the byte array, since the text file contains code from another programming language that likely won't be changing in the future.
                This approach sounds like a train wreck waiting to happen. Don't to it.

                What have you got against a simple loop?
                • 5. Re: Maximum Size of Scanner String
                  908528
                  Yes, your approach is the best here. I've just been programming in R a lot in the past 2 years, and everything there is vectorised, and people from that community hate loops.