7 Replies Latest reply on Apr 29, 2009 7:19 AM by EJP

    ProcessBuilder returns unknown error code 5

    843790
      Hi,

      We have a piece of code that executes external legacy executables. We have around 1000 unit tests throughout the suite that cause this piece of code to execute under almost exactly the same input/output/environmental conditions during each Continuous Integration and Nightly build. Occassionally, randomly (I dare to state randomly == once per day, at different times of day, sometimes on developer box, sometimes on build server, ie randomly), execution fails unexpectedly with Error Code 5. I and my colleagues have been looking into this for a couple of days, and we're battling to find out the true meaning of this "5" :-(

      What we've found so far, is some documentation on the MSDN for possibly the underlying Win32 "create" method that indicates the following:

      Return code     Description
      0 Successful Completion
      2 Access Denied
      3 Insufficient Privilege
      8 Unknown failure
      9 Path Not Found
      21 Invalid Parameter

      And nothing more than this in the Java docs. Can anyone help us?

      Our important ProcessBuilder bits looks as follows (sorry for the length - just proving that we think we covered all angles):

      private void initialiseAndStart(String[] executionDetails,
      File workingDirectory, Map<String, String> environmentParameters,
      String executableName, List<String> promptDelimiters, boolean runInCommandLine) throws IOException {

      // 16-bit executables no longer require support, all recompiled to 32-bit.
      // and above code in-lined to executionDetails only.
      String executableFilePath = executableName;
      File executableFile = new File(executableFilePath);
      if (!executableFile.exists()) {
      throw new FileNotFoundException(
      String.format(
      "Unable to launch program! Cannot find requested executable '%s'."
      + " Check your installation directory contents and "
      + "configuration settings",
      executableFilePath));
      }

      // Temporarily reinstate 16-bit support so that we can make progress
      String[] fullExecutionDetails;

      final String commandLineExe = "cmd.exe";
      final String executeAndExitFlag = "/c";
      final String newCommandLineWindow = "start";

      String[] preamble = new String[0];
      if (runInCommandLine) {
      preamble = new String[]{commandLineExe, executeAndExitFlag, newCommandLineWindow};
      } else if (isSixteenBitExecutable(executableFile)) {
      preamble = new String[]{commandLineExe, executeAndExitFlag};
      }

      fullExecutionDetails = ArrayUtil.flattenArraysDestructively(preamble, executionDetails);

      pb = new ProcessBuilder(fullExecutionDetails);

      // pass the environment parameters to the process builder as name value pairs
      Map<String, String> env = pb.environment();
      for (String key : environmentParameters.keySet()) {
      String value = environmentParameters.get(key);
      env.put(key, value);
      LOGGER.trace("Adding '" + key + "' = '" + value
      + "' to the external execution environment");
      }

      pb.directory(workingDirectory);

      StringBuilder fullExecutionString = new StringBuilder();
      for (String element : fullExecutionDetails) {
      if (element != null) {
      fullExecutionString.append(element).append(" ");
      }
      }
      processName = executableFile.getName();
      LOGGER.info("Running external program '" + processName
      + "' in directory '" + workingDirectory + "'");
      LOGGER.debug("Executing: " + fullExecutionString.toString());

      // start may not start immediately - but it looks like a value will not be returned until it has.
      process = pb.start();

      if (maximumExecutionSeconds > INFINITY_TIMEOUT) {
      ExternalProcessTimeoutKiller.createTimeoutKiller(maximumExecutionSeconds, this);
      }

      stdErrReader = new InterruptibleStreamReader(process
      .getErrorStream());
      stdOutReader = new InterruptibleStreamReader(process.getInputStream(),
      promptDelimiters);
      stdInWriter = new BufferedWriter(new OutputStreamWriter(process
      .getOutputStream()));
      }


      Looking forward to any response on this one!

      Best regards
      Higgser

      Edited by: higgser on Apr 14, 2009 11:10 AM

      The following exception is returned from the pb.start() line above:

      java.io.IOException: CreateProcess: D:\mydirectory\myexecutable.exe myparam error=5
           at java.lang.ProcessImpl.create(Native Method)
           at java.lang.ProcessImpl.<init>(ProcessImpl.java:81)
           at java.lang.ProcessImpl.start(ProcessImpl.java:30)
           at java.lang.ProcessBuilder.start(ProcessBuilder.java:451)
           at com.alstom.its.library.exec.internal.ExternalProcess.initialiseAndStart(ExternalProcess.java:362)
        • 1. Re: ProcessBuilder returns unknown error code 5
          843790
          Error code 5 is almost certainly coming from your exe file. Also, this code is suspect
          stdErrReader = new InterruptibleStreamReader(process
          .getErrorStream());
          stdOutReader = new InterruptibleStreamReader(process.getInputStream(),
          promptDelimiters);
          stdInWriter = new BufferedWriter(new OutputStreamWriter(process
          .getOutputStream()));
          since you don't seem to process the three stream in 3 threads you have the potential for a deadlock. You should read the 4 sections of http://www.javaworld.com/javaworld/jw-12-2000/jw-1229-traps.html and implement the recommendations. Although this 'traps' article is primarily concerned with Runtime.exec(), Runtime.exec() uses ProcessBuilder behind the scenes and the 'traps' are the same.

          Could it be that your exe is detecting a deadlock and terminating itself with error code 5?
          • 2. Re: ProcessBuilder returns unknown error code 5
            843790
            The output and error Streams are actually own threads, but the input reader isn't.

            public class InterruptibleStreamReader extends Thread {
            ...
            }

            Finally we found a more complete list of Win32 error codes:

            http://help.netop.com/support/errorcodes/win32_error_codes.htm

            Code     Description      Name
            0      The operation completed successfully.      ERROR_SUCCESS
            1      Incorrect function.      ERROR_INVALID_FUNCTION
            2      The system cannot find the file specified.      ERROR_FILE_NOT_FOUND
            3      The system cannot find the path specified.      ERROR_PATH_NOT_FOUND
            4      The system cannot open the file.      ERROR_TOO_MANY_OPEN_FILES
            5      Access is denied.      ERROR_ACCESS_DENIED
            • 3. Re: ProcessBuilder returns unknown error code 5
              796085
              That list doesn't square with your earlier one. Generally the "net helpmsg" command can convert an error code into a message - assuming it's a system generated error code. Your process might have just done exit(5) or something.
              C:\>net helpmsg 5
              Access is denied.
              
              C:\>
              • 4. Re: ProcessBuilder returns unknown error code 5
                843790
                Sorry for the accidental double post...

                @sabre - thanks for the good article and tips!

                The output and error Streams are actually own threads (sorry, hidden details I thought irrelevant to post previously).

                public class InterruptibleStreamReader extends Thread {
                ...
                }

                The input reader is deliberately not a separate thread as we did not want the extra issues of trying to synchronize between a legacy executable prompting for user inputs, and supplying them. Do you think this is really necessary - to date we have wrapped around 30 executables with this code, and they all run perfectly well, except for this random issue we've encountered. The executables which are giving us this random error=5 response is a very simple converter, and no special prompted outputs/inputs are required.

                We found a more complete list of Win32 error codes on http://help.netop.com/support/errorcodes/win32_error_codes.htm, which suggest that we sometimes have an access problem:

                Code     Description      Name
                0      The operation completed successfully.      ERROR_SUCCESS
                1      Incorrect function.      ERROR_INVALID_FUNCTION
                2      The system cannot find the file specified.      ERROR_FILE_NOT_FOUND
                3      The system cannot find the path specified.      ERROR_PATH_NOT_FOUND
                4      The system cannot open the file.      ERROR_TOO_MANY_OPEN_FILES
                5      Access is denied.      ERROR_ACCESS_DENIED
                ....

                This got us to thinking again that perhaps the while the previous executable (we run tool chains) has completed and exited successfully, the large binary file that was created by the previous legacy executable may still somehow (thanks to Windows) be flushing to the file system, and Windows then denies read access until it is happy. Is this a plausible theory?

                Best regards
                Higgser
                • 5. Re: ProcessBuilder returns unknown error code 5
                  843790
                  higgser wrote:
                  Sorry for the accidental double post...

                  @sabre - thanks for the good article and tips!

                  The output and error Streams are actually own threads (sorry, hidden details I thought irrelevant to post previously).

                  public class InterruptibleStreamReader extends Thread {
                  ...
                  }
                  I can't see how your InterruptibleStreamReader works in the context you have shown. You don't seem to process the content of the streams. But ...
                  as we did not want the extra issues of trying to synchronize between a legacy executable prompting for user inputs, and supplying them. Do you think this is really necessary - to date we have wrapped around 30 executables
                  The input reader is deliberately not a separate thread
                  One can process one of the streams in the thread that is being used to construct the Process so in general one only needs two extra threads.
                  with this code, and they all run perfectly well, except for this random issue we've encountered. The executables which are giving us this random error=5 response is a very simple converter, and no special prompted outputs/inputs are required.

                  We found a more complete list of Win32 error codes on http://help.netop.com/support/errorcodes/win32_error_codes.htm, which suggest that we sometimes have an access problem:

                  Code     Description      Name
                  0      The operation completed successfully.      ERROR_SUCCESS
                  1      Incorrect function.      ERROR_INVALID_FUNCTION
                  2      The system cannot find the file specified.      ERROR_FILE_NOT_FOUND
                  3      The system cannot find the path specified.      ERROR_PATH_NOT_FOUND
                  4      The system cannot open the file.      ERROR_TOO_MANY_OPEN_FILES
                  5      Access is denied.      ERROR_ACCESS_DENIED
                  ....

                  This got us to thinking again that perhaps the while the previous executable (we run tool chains) has completed and exited successfully, the large binary file that was created by the previous legacy executable may still somehow (thanks to Windows) be flushing to the file system, and Windows then denies read access until it is happy. Is this a plausible theory?
                  Possible! To test the theory, you could try delaying execution by say 1 second but who knows!
                  • 6. Re: ProcessBuilder returns unknown error code 5
                    843790
                    The frequency of the random access denied error cound be reduced by adding additional delays until the input file is readable (File.canRead() returns true). This doesn't solve all problems yet, but further debugging must be postponed.
                    • 7. Re: ProcessBuilder returns unknown error code 5
                      EJP
                      If you're dependent on the previous process finishing you should be calling Process.waitFor() somewhere.

                      One of those lists of error codes is for the NetXXX APIs: doesn't apply to ProcessBuilder.start().