12 Replies Latest reply: Feb 17, 2010 2:32 AM by 843798 RSS

    OutOfMemory: PermGen space using the URLClassLoaer

    843798
      Though this would be the closest matching forum to put this topic, sine there is some reflection involved

      java.lang.OutOfMemoryError: PermGen
      space
      at java.lang.ClassLoader.defineClass1(Native Method)
      at java.lang.ClassLoader.defineClass(ClassLoader.java:620)
      at
      java.security.SecureClassLoader.defineClass(SecureClassLoader.java:124)
      at java.net.URLClassLoader.defineClass(URLClassLoader.java:260)
      at java.net.URLClassLoader.access$100(URLClassLoader.java:56)
      at java.net.URLClassLoader$1.run(URLClassLoader.java:195)
      at java.security.AccessController.doPrivileged(Native Method)
      at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
      at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:579)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
      at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
      at java.lang.Class.getDeclaredMethods0(Native Method)
      at java.lang.Class.privateGetDeclaredMethods(Class.java:2395)
      at java.lang.Class.getDeclaredMethods(Class.java:1763)

      Below is the snippet using the URLClassLoader
      ....
      ClassLoader prevCl = Thread.currentThread().getContextClassLoader();
      URL url = // defines a bunch of depend api jars related to JAXB and Initial Context
      ClassLoader urlCl = URLClassLoader.newInstance(url, prevCl);
      Thread.currentThread().setContextClassLoader(urlCl);
      urlCl.loadclass(classname);


      where classname refers to the point of entry into the class say "JAXBLoader" which process an xml by unmarshalling into java object and makes a webservice call.Once after completing the above process successfully there is increase in number of classes loaded ( in the order of 100,000) and repeating this process with different or same xml causes a PermGen space OutofMemory.

      Has anyone come across this issue using the URLCLassLoader by loading the class files dynamically ?.

      -thanks !
        • 1. Re: OutOfMemory: PermGen space using the URLClassLoaer
          EJP
          OutOfMemory only ever has one cause, but why are you calling loadClass() and then throwing away the result?
          • 2. Re: OutOfMemory: PermGen space using the URLClassLoaer
            843798
            @ejp: I am sorry , not quite sure I understood ur point.But what I am trying to do is load the class files dynamically from the jar files, like a web/app sever container does.Though it is effective, I run into this outofmemory (PermGen space) after few iterations.
            • 3. Re: OutOfMemory: PermGen space using the URLClassLoaer
              EJP
              urlCl.loadclass(classname);
              why are you calling loadClass() and then throwing away the result?
              not quite sure I understood your point.
              I don't see why not, it's pretty simple. You are calling urlCl.loadClass() and throwing away the result. You are loading a class but never using it. What is the point? If you're trying to run out of memory, this would be a good way to do it, but it has no other point that I can see.
              • 4. Re: OutOfMemory: PermGen space using the URLClassLoaer
                843798
                I got ur question now. i didn't add the code that would have added some information

                I am creating an instance from the loaded class like below

                Class jaxbloaderclass = urlCl.loadclass(classname);
                JAXBLoader instance1 = jaxbloaderclass.newInstance();
                instance1.processor();



                public class JAXBLoader{

                ....

                processor(){
                code that does the marshalling, reflection etc using the JAXB API and JAXB Binding classes generated from XSD.
                }
                ...

                }

                thanks!
                • 5. Re: OutOfMemory: PermGen space using the URLClassLoaer
                  EJP
                  So why are you creating a new URLClassLoader every time you do this? Are the URLs changing?
                  • 6. Re: OutOfMemory: PermGen space using the URLClassLoaer
                    843798
                    Yes. the URLs (jar files) would be changing.
                    • 7. Re: OutOfMemory: PermGen space using the URLClassLoaer
                      796085
                      Do you really mean 100,000 classes loaded? Or do you mean 100,000 instances created. PermGen is used for (among other things) class metadata. If you really are loading 100,000 distinct classes then you may well need to increase your PermGen space.
                      • 8. Re: OutOfMemory: PermGen space using the URLClassLoaer
                        843798
                        I monitored using the jconsole which shows the no of classes loaded ,unload and total no if classes loaded.For 1 iteration, i would see an increase of 100,000 classes in the classes loaded and total classes loaded.Increasing the permgen space size at startup might only be a temporary fix , since we are not fixing the flaw and eventually would run into the same issue after a sometime.
                        • 9. Re: OutOfMemory: PermGen space using the URLClassLoaer
                          DrClap
                          jopensource wrote:
                          @ejp: I am sorry , not quite sure I understood ur point.But what I am trying to do is load the class files dynamically from the jar files, like a web/app sever container does.
                          The difference is that an application server actually uses the classes after it loads them, instead of just loading them and ignoring them. Is there a reason you are doing that?
                          • 10. Re: OutOfMemory: PermGen space using the URLClassLoaer
                            796311
                            And an app server will unload classes being replaced by a redistribution of an application. Well, it is not perfect, but it does do so even though it does increase the size while it does it since garbage collection is involved.
                            • 11. Re: OutOfMemory: PermGen space using the URLClassLoaer
                              jschellSomeoneStoleMyAlias
                              jopensource wrote:
                              I monitored using the jconsole which shows the no of classes loaded ,unload and total no if classes loaded.For 1 iteration, i would see an increase of 100,000 classes in the classes loaded and total classes loaded.Increasing the permgen space size at startup might only be a temporary fix , since we are not fixing the flaw and eventually would run into the same issue after a sometime.
                              Computers are fixed size systems. There is no way to load an infinite number of anything into them.

                              If you need to load 100,000 different classes or more then your choices are
                              1. Get a really, really big machine with a 64 bit VM.
                              2. Don't load that many classes.
                              since we are not fixing the flaw
                              Sounds like a bad idea. Price a 64 bit system with 64 CPUs and 32 terabytes of physical memory and tell whoever came up with the idea that fixing the "flaw" isn't a good idea that you will need 4 of those systems to support the current system: 2 for prod, 1 for QA and 1 for development.
                              • 12. Re: OutOfMemory: PermGen space using the URLClassLoaer
                                843798
                                jopensource wrote:
                                Yes. the URLs (jar files) would be changing.
                                Why? I doubt you're using entirely separate JAX-WS implementations, so what jars are changing?

                                I think you might get a much better situation if you used one ClassLoader to load the (unchanging) library classes and hold on to that.

                                Then use a separate ClassLoader to load the jar files that change (and use the previous ClassLoader as its parent). This way you'll only load changed classes, when the jar files change.