6 Replies Latest reply: May 8, 2013 10:24 AM by sabre150 RSS

    App started by ProcessBuilder can't find extracted native libraries

    1007721
      Hi all,

      I have a strange issue with my application.

      This is what I want:

      1: My Java bootloader (main class that starts another application with ProcessBuilder) search on the classpath for config files containing startup properties. The bootloader sets environment variables and vm properties like LD_LIBRARY_PATH and the -Xmx and starts the real application.
      2: The application extracts native libraries based on OS and Archtype to a temp folder based on a timestamp (LD_LIBRARY_PATH refers to that temp folder). The native libraries are packed in jar files, that are placed on the classpath.
      3: The application loads the native libraries. In my scenario there are 3 native libs. 1 of the files have a dependency to the other 2. (loadlibrary of the one use the LD_LIBRARY_PATH to locate the other 2 on Linux)

      This is working correctly at Windows and also on Linux when I did NOT use the ProcessBuilder but start the process directly (and set the LD_LIBRARY_PATH before starting the application).

      Under Linux (with using my Java bootloader) I have the problem that my application [ System.load(...) ] throws an exception that the related libraries can't be found when loading the library. But when I hardcode/reuse my temp folder, so that the files are already available when starting the bootloader, the loading will succeed. The LD_LIBRARY_PATH always reference to the correct path of the native libraries.

      I don't understand what is wrong? Any ideas?
        • 1. Re: App started by ProcessBuilder can't find extracted native libraries
          sabre150
          You are obviously not setting up LD_LIBRARY_PATH in ProcessBuilder correctly but without seeing the code I can't be more specific.
          • 2. Re: App started by ProcessBuilder can't find extracted native libraries
            1007721
            My console dumps (with some added comments). The environment variables seems to be set correctly.

            ### DUMP from JAVA application with generated temp folder -> comments in brackets [] added to dump

            TVDS_TMP_DIR=/tmp/tvds1895770185265633201 [GENERATED and set by process builder]

            LD_LIBRARY_PATH=/tmp/tvds1895770185265633201/native-libs [used System.getEnv("LD_LIBRARY_PATH")]

            Extracting /tmp/tvds1895770185265633201/native-libs/libMSPdtcc.so
            INFO: Loading /tmp/tvds1895770185265633201/native-libs/libMSPdtcc.so [not needed to load]

            Extracting /tmp/tvds1895770185265633201/native-libs/libMSPCoordinateConversionService.so
            INFO: Loading /tmp/tvds1895770185265633201/native-libs/libMSPCoordinateConversionService.so [not needed to load]

            Extracting /tmp/tvds1895770185265633201/native-libs/libjnimsp.so
            INFO: Loading /tmp/tvds1895770185265633201/native-libs/libjnimsp_ccs.so [dependencies are libMSPdtcc.so & libMSPCoordinateConversionService.so]
            ...
            Caused by: java.lang.UnsatisfiedLinkError: /tmp/tvds1895770185265633201/native-libs/libjnimsp_ccs.so: libMSPdtcc.so: cannot open shared object file: No such file or directory
                 at java.lang.ClassLoader$NativeLibrary.load(Native Method)
                 at java.lang.ClassLoader.loadLibrary1(ClassLoader.java:1939)
                 at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1864)
                 at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1825)
                 at java.lang.Runtime.load0(Runtime.java:792)
                 at java.lang.System.load(System.java:1059)
                 at site_extras.dependency_util.NativeLibLoader.loadLibrary(NativeLibLoader.java:58)
                 ... 31 more

            ### DUMP from JAVA application with hardcoded temp folder (files already available from previous run)

            TVDS_TMP_DIR=/tmp/tvds1895770185265633201 [hardcoded]

            LD_LIBRARY_PATH=/tmp/tvds1895770185265633201/native-libs

            INFO: Loading /tmp/tvds1895770185265633201/native-libs/libMSPdtcc.so
            INFO: Loading /tmp/tvds1895770185265633201/native-libs/libMSPCoordinateConversionService.so
            INFO: Loading /tmp/tvds1895770185265633201/native-libs/libjnimsp_ccs.so

            [NO EXCEPTION]
            • 3. Re: App started by ProcessBuilder can't find extracted native libraries
              sabre150
              Sorry but I see no Java code so I cannot compare with my working code to see what is different!
              • 4. Re: App started by ProcessBuilder can't find extracted native libraries
                1007721
                I do something like this with the ProcessBuilder (code is a little bit more complex because I add also some VM flags like -Xmx that are read from a config file)
                String projectTmpLocation = createTempDir();
                
                ProcessBuilder processBuilder = new ProcessBuilder(JAVA_CMD, "-cp", CLASSPATH);
                processBuilder.inheritIO();
                
                Map<String, String> environment = processBuilder.environment();
                environment.put("TVDS_TMP_DIR", projectTmpLocation);
                environment.put("LD_LIBRARY_PATH", projectTmpLocation + File.separator + "native-libs");
                
                List<String> commandList = processBuilder.command();
                commandList.add(Application.class.getName());
                          
                processBuilder.start();
                • 5. Re: App started by ProcessBuilder can't find extracted native libraries
                  1007721
                  Hi,

                  I found a solution for my problem. The directory of the native libs should be created before the ProcessBuilder.start() (new process) is called.

                  In the old situation I only set the LD_LIBRARY_PATH to something like /tmp/tvds324632879/native-libs and the real application create the director. When I create the /tmp/tvds324632879/native-libs in my bootloader, native libs loading will be successfull.
                  • 6. Re: App started by ProcessBuilder can't find extracted native libraries
                    sabre150
                    Thanks for posting your solution. I have just created a self contained test harness that worked every time BUT it just happens to make sure the temp directory exists and copies all the files into the directory before doing anything with ProcessBuilder! It never occurred to me to do it the other way round since the start() method executes the chained program!