1 2 3 Previous Next 34 Replies Latest reply: Dec 13, 2009 11:23 PM by 843810 RSS

    javac compiles very slow using jdk 1.6

    843810
      hi,

      we are trying to migrate our projects from java 1.5 to java 1.6 and have the problem that compile times are decreasing by factor 2-10 on our larger projects. first I was thinking that maven or the maven-compiler-plugin is the problem, but then I tried the same just by using javac for compiling.


      As Example I extracted the classpath and sourcepath used by maven and build argument files for javac:

      javac @options.txt @classes.txt

      the classpath contains 56 jar-files (with full path)
      the "classes.txt" contains 447 source-files (also listed with full path)

      For compiling this Java 5 needs betwwen 7 and 8 seconds while Java 6 needs 52 seconds! ;-(

      Environment is:
      Windows XP SP 2
      JDK 6 Update 13
      (in Comparison to JDK 5 Update 11)


      Is there a problem in javac 1.6 with reading long argument files or using long full paths to source files?
      Does anybody else suffer from this problem? Is there a workaround?

      As already said actually we use maven for the compilation, so switching to ant is not want we want to do....

      Thanks,
      Jochen

      P.S.: I also posted in the Nabble Maven-Users Forum: [http://n2.nabble.com/Slow-maven-compile-after-upgrading-JDK-from-1.5-to-1.6-tc2868702.html]
        • 1. Re: javac compiles very slow using jdk 1.6
          jschellSomeoneStoleMyAlias
          Is there a workaround?
          One of course would be to structure your build/code such that developers seldom need to work on the application versus working on a a subsystem of the application.
          • 2. Re: javac compiles very slow using jdk 1.6
            843810
            I did some more tests to isolate the problem.

            First I tried by just compiling one file of the project by just leaving one entry in the "classes.txt" file. And it was SLOW (10 seconds for one .java-file)!!!

            So the problem must be the classpath containing alle the jar files.



            Then I did a test using ANT (ant's javac task).
            I copied all jar files to a "lib" directory and used this path for the javac task:
            <path id="antlike.class.path">
                      <fileset dir="./lib">
                           <include name="**/*.jar" />
                      </fileset>
                 </path>
            Compiling that way was FAST, so I don't think that there's a problem with all the jar files our code depends on.


            So in my next try with ANT I build the classpath the same way as I did in @options.txt argument file and like MAVEN does. I included all jar files to the path by specifying the whole path:
            <path id="mavenlike.class.path">
                      <fileset dir="c:/Daten/repository">               
                           <include name="commons-lang/commons-lang/2.3/commons-lang-2.3.jar" />
                           <include name="commons-beanutils/commons-beanutils/1.7.0/commons-beanutils-1.7.0.jar" />
                                    ........     54 more includes
                            </fileset>
                 </path>
            And this way again, it was very SLOW!

            Hence, I'm quite shure that there is a bug in javac of java1.6, which causes problems when passing many long path strings to compiler using the -classpath option.
            I just wonder that nobody else seems to have the problem (at least I didn't find a hint on the www).
            How can I "officially" report that bug? Or has anybody already found a solution?
            • 3. Re: javac compiles very slow using jdk 1.6
              jschellSomeoneStoleMyAlias
              j_ri wrote:
              I just wonder that nobody else seems to have the problem (at least I didn't find a hint on the www).
              Because very few people are willing to put up with the build environment that you described. I know would scream loudly if you told me that the only way I could work on a subsystem was by using 50+ jars.
              How can I "officially" report that bug? Or has anybody already found a solution?
              1. Replicate it with an example.
              2. Post it to the bug database.
              3. Find another solution because they are unlikely to fix it soon unless you have a sales contract with them.
              • 4. Re: javac compiles very slow using jdk 1.6
                EJP
                Hence, I'm quite shure that there is a bug in javac of java1.6, which causes problems when passing many long path strings to compiler using the -classpath option.
                Why? The command line would have been more or less identical in both cases. Javac doesn't understand *.jar: ant would have expaneded that into the same list.
                • 5. Re: javac compiles very slow using jdk 1.6
                  843810
                  No, it's not the same list.

                  In the one case, the "original" paths were used:
                  c:\Daten\repository\commons-lang\commons-lang\2.3\commons-lang-2.3.jar
                  c:\Daten\repository\commons-beanutils\commons-beanutils\1.7.0\commons-beanutils-1.7.0.jar
                  c:\Daten\repository\commons-betwixt\commons-betwixt\0.8\commons-betwixt-0.8.jar
                  c:\Daten\repository\jdom\jdom\1.0\jdom-1.0.jar
                  ........


                  In the *.jar case the list would be:
                  c:\test\lib\commons-lang-2.3.jar
                  c:\test\lib\commons-beanutils-1.7.0.jar
                  c:\test\lib\commons-betwixt-0.8.jar
                  c:\test\lib\jdom-1.0.jar
                  ....

                  So,in the second case all jars are in the same folder and the path-string is quite short. I just did the test to write every single include in the fileset instead of doing *.jar, for the example where all jar-files are in the "c:\test\lib\" directory. This way it's also fast.

                  Hence javac becomes only slow with jar-files lying in diffenet folders and long pathnames pointing to them (as it is the case with the Apache Maven2 buildsystem).


                  Meanwihile I reported the bug to sun.
                  • 6. Re: javac compiles very slow using jdk 1.6
                    EJP
                    It's rather difficult to believe in that as the cause of the problem.

                    Were the jars listed in the same order?
                    • 7. Re: javac compiles very slow using jdk 1.6
                      843810
                      yes, the order was/is the same.

                      I extracted the classpath from the maven build logfile and manipulated the paths for my different tests on the commandline and with ANT, but I always kept the order of the jars.

                      Actually I also couldn't believe that this is the cause of the problem, but I can't see another possibility anymore.
                      I wonder what they changed since Java 1.5, because with the "old " javac all tests run smoothly and fast.
                      • 8. Re: javac compiles very slow using jdk 1.6
                        jschellSomeoneStoleMyAlias
                        ejp wrote:
                        Hence, I'm quite shure that there is a bug in javac of java1.6, which causes problems when passing many long path strings to compiler using the -classpath option.
                        Why? The command line would have been more or less identical in both cases. Javac doesn't understand *.jar: ant would have expaneded that into the same list.
                        Could have changed the way that classes are loaded from jars in the VM. The compiler is built on the VM so it would be impacted in the same way.
                        • 9. Re: javac compiles very slow using jdk 1.6
                          EJP
                          Could have changed the way that classes are loaded from jars in the VM.
                          Only if the JARs were listed in a different order. I asked that question and the answer was no. I find it hard to believe that the length of a JAR filename has a performance impact.
                          • 10. Re: javac compiles very slow using jdk 1.6
                            843810
                            >
                            Because very few people are willing to put up with the build environment that you described. I know would scream loudly if you told me that the only way I could work on a subsystem was by using 50+ jars.
                            Actually I think a lot of people using Maven2 have a lot of jars in their classpath, because all transitive dependencies get in the classpath, too. If you depend (directly) on about 10 jars...some of these jars depend on other jars. I don't know why Maven puts them in the compile classpath as many of the jars are "only" needed at runtime.
                            But this would be another discussion....;-)
                            • 11. Re: javac compiles very slow using jdk 1.6
                              843810
                              ejp wrote:
                              Only if the JARs were listed in a different order. I asked that question and the answer was no. I find it hard to believe that the length of a JAR filename has a performance impact.
                              It is not the name of "a JAR filename", but the name of many JAR names.

                              But as I find it hard to believe myself, what else could be the problem?
                              - Navigation in the filesystem (each jar is placed in a different folder)?
                              - parsing the very long commandline?
                              - different default memory settings between 1.5 and 1.6?
                              - or is javac itself is reordering the JARs before passing them to the actual compiler?
                              • 12. Re: javac compiles very slow using jdk 1.6
                                EJP
                                It is not the name of "a JAR filename", but the name of many JAR names.
                                It is the length of all the JAR filenames.
                                - Navigation in the filesystem (each jar is placed in a different folder)?
                                - parsing the very long commandline?
                                Neither of those would make a difference measured in minutes.
                                - different default memory settings between 1.5 and 1.6?
                                The total length of the JAR filenames is insignificant compared to the total length of all the Java code that has to be read and the .class data that has to be written.
                                - or is javac itself is reordering the JARs before passing them to the actual compiler?
                                Javac is the actual compiler. Why would it do that?

                                You said before that the ordering was the same, but one of the commands used a *.jar format. So how do you know the ordering was the same?

                                Are all these JAR files on a local disk?
                                • 13. Re: javac compiles very slow using jdk 1.6
                                  843810
                                  You said before that the ordering was the same, but one of the commands used a {noformat}*.jar{noformat} format. So how do you know the ordering was the same?
                                  This was only one of the tests. after I noticed that ordering could be a problem I replaced the test by doing it only with putting each single file as include to the ANT script.{noformat}With Maven and on the commandline with the argument file I never used something like *.jar{noformat}
                                  Are all these JAR files on a local disk?
                                  yes, they are on a local disc. (disabling antivirus software, also didn't have a noticeable effect)


                                  if I find some time today, I try to do a build using OpenJDK for comparison

                                  Edited by: j_ri on May 20, 2009 7:47 AM
                                  • 14. Re: javac compiles very slow using jdk 1.6
                                    jschellSomeoneStoleMyAlias
                                    ejp wrote:
                                    Could have changed the way that classes are loaded from jars in the VM.
                                    Only if the JARs were listed in a different order. I asked that question and the answer was no. I find it hard to believe that the length of a JAR filename has a performance impact.
                                    Huh? Are we discussing different things?

                                    The VM could have change the way that it loads jars from 1.5 to 1.6. A trivial example would be that in 5 it could have parsed the class path string each time it looked for a jar where in 6 is keeps a parsed list around. (Not saying that is what happening but rather an example of a possible change.)
                                    1 2 3 Previous Next