1 2 3 Previous Next 34 Replies Latest reply: Dec 13, 2009 11:23 PM by 843810 Go to original post RSS
      • 15. Re: javac compiles very slow using jdk 1.6
        EJP
        I would need a much more plausible example than that. That example wouldn't explain a difference of many minutes, any more than anything else that's been suggested in this thread.
        • 16. Re: javac compiles very slow using jdk 1.6
          jschellSomeoneStoleMyAlias
          ejp wrote:
          I would need a much more plausible example than that. That example wouldn't explain a difference of many minutes, any more than anything else that's been suggested in this thread.
          Not sure that I claimed that it would. Certainly not my intent. It was provided of an example of how the VM could have changed the load process between versions.

          My presumption is that the OP is in fact seeing a significant difference. And a repeatable one. Thus some explanation must exist. I doubt that Sun tests the VM (either in 5 or 6) by loading that many jar files into it to see what happens. Thus something certainly could be wrong.
          • 17. Re: javac compiles very slow using jdk 1.6
            EJP
            Not sure that I claimed that it would.
            I'm not sure why you raised the issue 'could have changed' then.
            Thus some explanation must exist.
            Of course.
            Thus something certainly could be wrong.
            Of course. Is somebody saying something different?

            The question is what could be wrong, or different, that would explain the observed phenomena.

            Not just some arbitrary difference between Java 5 and 6.
            • 18. Re: javac compiles very slow using jdk 1.6
              jschellSomeoneStoleMyAlias
              ejp wrote:
              Not sure that I claimed that it would.
              I'm not sure why you raised the issue 'could have changed' then.
              At this point in time I have no idea what you intended to mean in reply #4.

              The OP suggested that the only change was 1.5 to 1.6. And from there the time to compile increased signficantly. Thus there was a bug.

              Based on the specification the conclusion that the OP reached was correct.

              It certainly appeared to me that you were questioning that conclusion.

              You might have been questioning the conclusion that it had to do with using classpath, but I fail to see how that could not be a valid conclusion. After all no one else has reported problems and that is one really big classpath in my experience.
              • 19. Re: javac compiles very slow using jdk 1.6
                EJP
                The OP understood my reply #4 all right, and he gave a series of cogent responses.
                Based on the specification the conclusion that the OP reached was correct.
                I don't know what 'specification' that might be, but the conclusions that have been stated here are implausible, and the OP said so himself. The logical inference from that is that we don't have all the data.

                Sun compile the JDK every night. It is a pretty huge compilation, and they would be the first to notice such a major regression. It could happen: it would be surprising. I would prefer a more plausible explanation. I don' t have one, but then I don't believe I have all the relevant information either.
                After all no one else has reported problems
                Exactly so.
                and that is one really big classpath in my experience.
                It's a lot of JAR files. But why would it be faster if they have short names rather than long names? and/or are in the same directory intead of multiple directories? This is the point at issue. The I/O activity on the JAR files will exceed all use of the command line and the actual filenames by many orders of magnitude. So I find it hard to believe the filename length or the directories can have anything to do with it.

                NB the concrete change you suggested about classloading would make it better. Not many minutes worse. If you can come up with an actual change between 1.5 and 1.6 that would explain this it would be very welcome. I'll follow the progress of the OP's bug report with interest.

                @OP: you might consider using java -i on some of the JAR files. At the worst, failing a plausible understanding & resolution of the issue, you can at least change the build to copy all the JAR files into a single temp directory and build the classpath from there, as you did above.
                • 20. Re: javac compiles very slow using jdk 1.6
                  jschellSomeoneStoleMyAlias
                  ejp wrote:
                  The OP understood my reply #4 all right, and he gave a series of cogent responses.
                  Based on the specification the conclusion that the OP reached was correct.
                  I don't know what 'specification' that might be, but the conclusions that have been stated here are implausible, and the OP said so himself. The logical inference from that is that we don't have all the data.
                  Ok.
                  Sun compile the JDK every night. It is a pretty huge compilation, and they would be the first to notice such a major regression. It could happen: it would be surprising. I would prefer a more plausible explanation. I don' t have one, but then I don't believe I have all the relevant information either.
                  I doubt it is that big. If they recompiled the entire Sun java base as one single compile then maybe.
                  After all no one else has reported problems
                  Exactly so.
                  and that is one really big classpath in my experience.
                  It's a lot of JAR files. But why would it be faster if they have short names rather than long names? and/or are in the same directory intead of multiple directories? This is the point at issue. The I/O activity on the JAR files will exceed all use of the command line and the actual filenames by many orders of magnitude. So I find it hard to believe the filename length or the directories can have anything to do with it.
                  Thus back to whether nothing changed but the compiler.
                  NB the concrete change you suggested about classloading would make it better. Not many minutes worse. If you can come up with an actual change between 1.5 and 1.6 that would explain this it would be very welcome. I'll follow the progress of the OP's bug report with interest.
                  Yes but I was only providing a theoretical example to emphasize the point. And certainly in the past Sun has implemented changes that should have made something faster, and instead in exteme cases, made it substantially worse.
                  • 21. Re: javac compiles very slow using jdk 1.6
                    843810
                    Meanwhile I got an answer to my bug report:
                    .....
                    We have determined that this report is a new bug and entered the bug into our internal bug tracking system under Bug Id: 6843751.

                    You can monitor this bug on the Java Bug Database at http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6843751.
                    .....
                    your suggestion
                    @OP: you might consider using java -i on some of the JAR files. At the worst, failing a plausible understanding & resolution of the issue, you can at least >change the build to copy all the JAR files into a single temp directory and build the classpath from there, as you did above.
                    to put all the jar files in a single directory won't work for us. we have about 20 projects organized in a maven2 (with cruisecontrol, etc.) build structure...and all of theses projects have between 5 and 15 submodules.

                    Since Sun determined that this is a bug, I think that we'll wait with our migration `til it is fixed.....
                    • 22. Re: javac compiles very slow using jdk 1.6
                      jschellSomeoneStoleMyAlias
                      j_ri wrote:
                      Meanwhile I got an answer to my bug report:
                      .....
                      We have determined that this report is a new bug and entered the bug into our internal bug tracking system under Bug Id: 6843751.

                      You can monitor this bug on the Java Bug Database at http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6843751.
                      .....
                      Until you see something in the section that says "Evaluation" it doesn't mean much. All the above says is that they have accepted it for evaluation.
                      Since Sun determined that this is a bug, I think that we'll wait with our migration `til it is fixed.....
                      Unless you have a sales contract with Sun and push via your sales rep if it was me I wouldn't count on that strategy. There are bugs in there that are trivial to fix with known solutions and yet haven't been fixed for years.
                      • 23. Re: javac compiles very slow using jdk 1.6
                        843810
                        Unless you have a sales contract with Sun and push via your sales rep if it was me I wouldn't count on that strategy. There are bugs in there that are trivial to fix with known solutions and yet haven't been fixed for years.
                        Thanks for the hint. The only strategy for us is just to compile modules with JDK 6 which really use new features and compile everything else with JDK 5. Perhaps our Nightly-Build can completely use JDK 6 ( if the night is long enough;-) ). But that will be quite a lot of configuration work with Maven.
                        Another Solution could be to change the way Maven assembles the classpath. Maybe there's a way that only the direct dependencies and not all transitive dependencies find their way in the classpath.

                        I'm curious if other people get the same problems when they "migrate" their larger projects to JDK 6.
                        • 24. Re: javac compiles very slow using jdk 1.6
                          jschellSomeoneStoleMyAlias
                          j_ri wrote:
                          I'm curious if other people get the same problems when they "migrate" their larger projects to JDK 6.
                          Since 1.6 has been out for a few years and you are the only one that has such a problem I would guess not.
                          I believe I already mentioned that the way you are set up seems completely unworkable to me, and I would guess that other "large" projects have structured their builds so that they don't attempt to rebuild the world via a single build. Thus no problem.
                          • 25. Re: javac compiles very slow using jdk 1.6
                            843810
                            Since 1.6 has been out for a few years and you are the only one that has such a problem I would guess not.
                            I agree that 1.6 is out for a few years.
                            But I don't agree that many larger projects (at least projects running on JEE servers) are already using it.
                            We have 3rd party software which is not yet 1.6 certified.
                            And, for exapmle, if you take a look at the JBoss Application Server release notes, they say that you can run the application server on 1.6 but "It should be noted however that the Java 6 compiled distribution of JBoss AS 5 is still in experimental stage."


                            I believe I already mentioned that the way you are set up seems completely unworkable to me, and I would guess that other "large" projects have structured their builds so that they don't attempt to rebuild the world via a single build. Thus no problem.
                            I think, you misunderstood me. Our projects are structured very well; in a lot of small pieces (JARs). We rebuild all pieces every night step by step, project by project via "Cruisecontrol" and "Apache Maven2" (I think this is a common continuous integration method).
                            Our problem is not that nightly build. The problem is that a developer only works on one "little" JAR so that he doesn't have to compile a lot of files (the dependencies are all there from the night before), but now needs a very long time for the building of his small artifact. As the process of compiling and (re-)deploying the artifact into a JEE container is already slow, the slow compilation time is not acceptable.


                            But before this post goes off topic, I try further to find a workaround;-)
                            • 26. Re: javac compiles very slow using jdk 1.6
                              843810
                              I think I found the cause for our problem.

                              All the time I have overlooked that the debug output of javac showed a completely different "[search path for class files: .....]" with JDK 1.5 and 1.6.

                              With Java 1.5 I have for example the entry

                              c:\Daten\repository\lbank\fiola\fiola-common\fiola-value\2.5.5\fiola-value-2.5.5.jar

                              This is what I put in the classpath argument.



                              With Java 1.6 javac suddenly prints out

                              c:\Daten\repository\lbank\fiola\fiola-common\fiola-value\2.5.5\fiola-value-2.5.5.jar
                              c:\Daten\repository\lbank\fiola\fiola-common\fiola-value\2.5.5\commons-lang-2.3.jar
                              c:\Daten\repository\lbank\fiola\fiola-common\fiola-value\2.5.5\jdom-1.0.jar
                              c:\Daten\repository\lbank\fiola\fiola-common\fiola-value\2.5.5\commons-betwixt-0.8.jar
                              c:\Daten\repository\lbank\fiola\fiola-common\fiola-value\2.5.5\commons-beanutils-core-1.7.0.jar
                              c:\Daten\repository\lbank\fiola\fiola-common\fiola-value\2.5.5\commons-digester-1.7.jar
                              .......
                              c:\Daten\repository\commons-lang\commons-lang\2.3\commons-lang-2.3.jar

                              First I didn't know why
                              c:\Daten\repository\lbank\fiola\fiola-common\fiola-value\2.5.5\commons-lang-2.3.jar
                              shows up, because I put
                              c:\Daten\repository\commons-lang\commons-lang\2.3\commons-lang-2.3.jar
                              in the classpath ( and only this is correct.

                              Discussing this with a co-worker we found out that javac put that in the search path, because the "fiola-value-2.5.5.jar" contains a manifest which has class-path entries (for commons-lang-2.3.jar and a lot of others).

                              Javac now seems to interpret the Class-Path of the manifest.mf. That didn't happen with 1.5.

                              Is this a bug or a new feature?
                              In my opinion it makes njo sense because, the class-path of the manifest should have the correct paths for runtime and not for compile-time, isn't it?

                              I try to "clean-out" the class-paths from the manifest files to see if compile times are fast again. This shouldn't matter for our applications running on JBoss AS, but for our standalone clients this could be a problem.
                              • 27. Re: javac compiles very slow using jdk 1.6
                                EJP
                                I think I found the cause for our problem.
                                Well done.
                                In my opinion it makes njo sense because, the class-path of the manifest should have the correct paths for runtime and not for compile-time, isn't it?
                                No, where does it say that? A classpath is a classpath.
                                I try to "clean-out" the class-paths from the manifest files to see if compile times are fast again.
                                You would be better off removing all explicit mentions in your compilation classpath of .jar files that are already referenced via Class-Paths in the manifest.
                                • 28. Re: javac compiles very slow using jdk 1.6
                                  843810
                                  In my opinion it makes no sense because, the class-path of the manifest should have the correct paths for runtime and not for compile-time, isn't it?
                                  No, where does it say that? A classpath is a classpath.
                                  It says nowhere that's why I said "in my opinion";-)
                                  I don't know if you ever worked with the apache maven2 build system, they even have 4 different scopes for the dependencies compile, runtime, provided, test and that concept makes sense.

                                  I try to "clean-out" the class-paths from the manifest files to see if compile times are fast again.
                                  You would be better off removing all explicit mentions in your compilation classpath of .jar files that are already referenced via Class-Paths in the manifest.
                                  I don't think that will work for me. For example:
                                  I want to build a JAR which is not a simple JAR, but contains an EJB. This JAR will end up in an EAR for deployment in an application server. That EAR-file must contain other JARs my EJB-JAr relies on. Default in JEE5 is that the EAR has a "lib" folder in it where all JAR files in need in the whole application are in. So my EJB-JAR has a classpath entry in its manifest that points to "lib/example.jar" or just "example.jar".
                                  But this relative path is not the path where the compiler finds the dependency, because in my build environment I don't have all JARs I depend on flattened in just on directory (actually when you work with maven you have a maven repository with a quite complicated structure that even contains version numbers in the path; you can take a look at [Maven 2 Repo|http://repo1.maven.org/maven2/] to see what I mean).

                                  I just checked a few artifacts in the central maven repository. the JARs there don't seem to have Class-Path enttries in their manifest, so I think the best solution really is to have no CP in the manifest.


                                  Actually, the best thing would be:
                                  If I have the "- classpath" option is pointing to "c:\Daten\repository\commons-lang\commons-lang\2.3\commons-lang-2.3.jar"
                                  and my manifest Class-Path contains the relative path "lib/commons-lang-2.3.jar"
                                  then the compiler should notice that its the same file (or filename) and should ignore that manifest entry instead of resolving
                                  "lib/commons-lang-2.3.jar" to "<project directory>/lib/commons-lang-2.3.jar"

                                  In my example that caused the "search path for class files" to have 500 entries (most of them pointing to places where no JAR exists) instead of having 59 entries with 1.5!


                                  P.S.: My compilation just finished. It was almost as fast as before with JDK 1.5 (perhaps I didn't clean all jars....;-). So I have my workaround now.
                                  • 29. Re: javac compiles very slow using jdk 1.6
                                    jschellSomeoneStoleMyAlias
                                    j_ri wrote:
                                    Javac now seems to interpret the Class-Path of the manifest.mf. That didn't happen with 1.5.

                                    Is this a bug or a new feature?
                                    Probably depends on whether it was intentional or not.
                                    In my opinion it makes njo sense because, the class-path of the manifest should have the correct paths for runtime and not for compile-time, isn't it?
                                    Doesn't make sense to me either. The manifest is a runtime entity. Thus for example you might include a jdbc driver in the jar because the classes in that jar need it at runtime. The classes in the jar are already compiled so they certainly will not need the manifest for compiling. Not sure I would really see the need for one manifest inclusion to span to another jar even at run time either. Thus it wouldn't be clear why compile time would do that.

                                    But as mentioned the class is the class path. The Sun compiler just runs on top of the VM so no reason for it to differentiate.