6 Replies Latest reply: Apr 11, 2010 5:19 AM by EJP RSS

    InputStream from external Process does not ReadLine() when CRs occur

    807580
      Hi There,

      I am developing an application that needs to start external processes and read/parse their output.
      This is how I try do do it (I think it follows common examples):
      final String[] arguments = new String[] { ... };
      final Process proc = Runtime.getRuntime().exec(arguments);
      final BufferedReader input = new BufferedReader(new InputStreamReader(proc.getErrorStream()));
      String procLine;
      while ((procLine = input.readLine()) != null) {
        // do something with procLine
      }
      input.close();
      With most extrnal programs, this works fine.

      But I have one application that overwrites its own output by using Carriage Returns (CR) until it's done.
      Although ReadLine() is documented to treat CR, LF, or any combination of them as newline, ir teads only one large line at the end of the external program. This line contains the CR's.
      But I need to get every line seperately: The lines contain the completion percentage and I want to display that continously in my status bar (yes, the external program is started in a new thread).

      Any ideas ?

      Thanks and Regards,
      Markus
        • 1. Re: InputStream from external Process does not ReadLine() when CRs occur
          807580
          I'm not convinced that the problem is the "\r". Since you don't say which OS or which Java version I can't be sure but the following works as anticipated on Ubuntu 9.10 using 1.6.0_19.
          import java.io.BufferedReader;
          import java.io.InputStreamReader;
          
          public class Sabre20100411
          {
              public static void main(String[] args) throws Exception
              {
                  final String[] arguments =
                  {
                      System.getProperty("user.home") + "/work/dev/perl/runtime_test.pl"
                  };
                  final Process proc = Runtime.getRuntime().exec(arguments);
                  final BufferedReader input = new BufferedReader(new InputStreamReader(proc.getErrorStream()));
                  String procLine;
                  while ((procLine = input.readLine()) != null)
                  {
                      // do something with procLine
                      System.out.println(procLine);
                  }
                  input.close();
              }
          }
          with the perl program being
          #!/usr/bin/perl
          for ($i = 0; $i < 50000; $i++)
          {
              print STDERR  "Stderr $i\r"; 
              sleep 1;
          }
          It is my experience that, due to buffering in the process of the stdout stream, the problem you are experiencing is common in external programs that send output to stdout but NOT when they send their output to stderr where by default no buffering takes place.

          I was concerned that maybe perl was doing something special with the CR but if one runs the perl command from the command line and redirects the output to a file then a hex dump of the file looks like

          00000000 53 74 64 65 72 72 20 30 0D 53 74 64 Stderr 0.Std
          0000000C 65 72 72 20 31 0D 53 74 64 65 72 72 err 1.Stderr
          00000018 20 32 0D 53 74 64 65 72 72 20 33 0D 2.Stderr 3.
          00000024 53 74 64 65 72 72 20 34 0D 53 74 64 Stderr 4.Std
          00000030 65 72 72 20 35 0D 53 74 64 65 72 72 err 5.Stderr
          0000003C 20 36 0D 53 74 64 65 72 72 20 37 0D 6.Stderr 7.

          The presence of the 0D characters indicates the the CR is being output without being augmented by a LF.

          Is there any reason why you are not processing stdout as well as stderr?

          Edited by: sabre150 on Apr 11, 2010 6:22 AM

          I have just tested this on Windows XP with 1.6.0_18 and "Active State" perl and it works as expected.

          Since there is no #! equivalent in Windows, I had to change the command to :-
           final String[] arguments =
                  {
                     "perl", System.getProperty("user.home") + "/work/dev/perl/runtime_test.pl"
                  };
          • 2. Re: InputStream from external Process does not ReadLine() when CRs occur
            807580
            Yes, indeed ...
            With your perl test program, it works.
            But when I replace it by the program I run, it doesn't.
            flac -d test.flac
            I will have to take a closer loot at the output of flac...
            • 3. Re: InputStream from external Process does not ReadLine() when CRs occur
              EJP
              Although ReadLine() is documented to treat CR, LF, or any combination of them as newline
              No it isn't. 'A line is considered to be terminated by any one of a line feed ('\n'), a carriage return ('\r'), or a carriage return followed immediately by a linefeed.' Not 'any combination'.
              • 4. Re: InputStream from external Process does not ReadLine() when CRs occur
                807580
                Mad_Markus wrote:
                Yes, indeed ...
                With your perl test program, it works.
                But when I replace it by the program I run, it doesn't.
                flac -d test.flac
                I will have to take a closer loot at the output of flac...
                I have also tested with C# on Windows (csc) and on Linux (Mono) and both work without problems.

                I know nothing at all about flac so I can't help there.
                • 5. Re: InputStream from external Process does not ReadLine() when CRs occur
                  807580
                  I found it !
                  flac uses backspaces to overwrite its output.

                  Looks like I need to read single chars into my own buffer instead of using readLine().

                  Thanks and Regards,
                  Markus
                  • 6. Re: InputStream from external Process does not ReadLine() when CRs occur
                    807580
                    Mad_Markus wrote:
                    I found it !
                    flac uses backspaces to overwrite its output.
                    What I find sad is that the OP jumped to the conclusion that Java was not handling CR properly without any real evidence. This feature of 'flac' could have been deduced very quickly just by redirecting the output of 'flac' to a file and then displaying the file content in hex.