13 Replies Latest reply: Apr 23, 2010 12:09 AM by 843790 RSS

    Instantiation from serialized Class requires .class file in classpath?

    843790
      My basic question is....I would like to know if a Class object in isolation (i.e. a serialized Class object obtained over the network) contains all the information necessary to instantiate a new instance, or if the Class object is merely a summary of information for reflection purposes and instantiation still requires a class loader.

      My overall problem is...I have a client/server distributed system with 1 server and many clients. The system supports "plug-ins", which are simply dynamically loaded classes that implement a specific abstract class or interface. Plug-in classes are used by both the server and client processes.

      Since I have 1 server but many clients, for ease of use I would like to be able to drop a plug-in class file into my server's file system and have the server distribute the new class to all connected clients over the network using serialization. I would like to do this as simply as possible.

      Furthermore, since clients may need to run on secure machines or machines that potentially have no internet access or very restricted access, I do not want the clients to have to dynamically load the plug-in class using a URL class loader (I also do not want to have to set up my own "class server" to do this).

      I have tried instantiating a Class object for the class of the plug-in on the server, serializing this object, and passing it to the clients through a socket. The idea is that after de-serializing the class object at the clients I would then use the newInstance() method to instantiate a new plug-in class dynamically at the clients.

      However, when the clients de-serialize the Class object, at the point of de-serialization (in ObjectInputStream) they always seem to call the class loader looking for the plug-in class' class file. Note that this is before I've even attempted to call newInstance() on the class object.

      This implies that the Class object itself is not "stand-alone," and that the class still needs to be loaded in the traditional way, and therefore that the class file of the plug-in needs to be in the classpath of the clients, which defeats the whole purpose of having the server distribute the new class.

      I understand that I could actually serialize the bytes of the class file, ship the bytes over the network to my clients through a socket, and re-create the class file on the client side, writing it to the local file system in the classpath of the client. However, I would like to avoid this work if necessary.

      In [http://java.sun.com/javase/6/docs/api/java/lang/ClassLoader.html] it states the following:

      +However, some classes may not originate from a file; they may originate
      from other sources, such as the network, or they could be constructed by an
      application. The method defineClass,%20int,%20int%29] converts an array of bytes into an instance of class
      Class. Instances of this newly defined class can be created using[Class.newInstance|http://java.sun.com/javase/6/docs/api/java/lang/Class.html#newInstance%28%29].+

      If I'm understanding this statement correctly, although does assume the method of shipping the bytes of the class file over the network, it also +implies that the class Class that's created is all that's needed to call newInstance(), and that no local class loading using the class loader is necessary.

      Perhaps there is some difference between a Class object created this way (explicitly using class file bytes) and the Class object that I serialized?


        • 1. Re: Instantiation from serialized Class requires .class file in classpath?
          EJP
          Instantiation from serialized Class requires .class file in classpath?
          Yes.
          I would like to know if a Class object in isolation (i.e. a serialized Class object obtained over the network) contains all the information necessary to instantiate a new instance
          No.
          or if the Class object is merely a summary of information for reflection purposes and instantiation still requires a class loader.
          Yes.
          I understand that I could actually serialize the bytes of the class file, ship the bytes over the network to my clients through a socket, and re-create the class file on the client side, writing it to the local file system in the classpath of the client. However, I would like to avoid this work if necessary.
          You can't.
          ... and that no local class loading using the class loader is necessary.
          Err, this is the class loader Javadoc you are citing.
          Perhaps there is some difference between a Class object created this way (explicitly using class file bytes) and the Class object that I serialized?
          Yes.
          • 2. Re: Instantiation from serialized Class requires .class file in classpath?
            843790
            Thank you very much for your answers. The only bit I'm still confused about is the quote (pasted here again with 1 additional preceeding paragraph ) which again, is from the URL I pasted above:

            Normally, the Java virtual machine loads classes from the local file system in a platform-dependent manner. For example, on UNIX systems, the virtual machine loads classes from the directory defined by the CLASSPATH environment variable.

            However, some classes may not originate from a file; they may originate from other sources, such as the network, or they could be constructed by an application. The method defineClass converts an array of bytes into an instance of class Class. Instances of this newly defined class can be created using Class.newInstance.

            This really seems to imply that a class can be instantiated from a remote source without having the class file in the local classpath. Am I still interpreting this incorrectly? Or is this paragraph referring to some other mechanism entirely?
            • 3. Re: Instantiation from serialized Class requires .class file in classpath?
              843790
              To re-phrase my last comment and try to answer my own question, since the class object created in the manner described in the class loader documentation is different than a serialized class object (per your answer), then is it this difference that allows the former to be used to instantiate an object whereas the latter cannot?
              • 4. Re: Instantiation from serialized Class requires .class file in classpath?
                EJP
                The ClassLoader loads classes. Serialization only gives you objects for which you already have the required .class file loaded or loadable via the CLASSPATH. The fact that the object concerned is itself a Class doesn't change that.
                • 5. Re: Instantiation from serialized Class requires .class file in classpath?
                  843790
                  Sure, but then what does the 2nd paragraph in the quote above mean? Again, it seems to say you can create a class from a chunk of bytes you send over the network and then use that class to instantiate objects, unless I'm reading it incorrectly.

                  If you're saying that despite using defineClass to create the class you still need the .class file in your classpath locally, then what would be the point of shipping the .class file's bytes over the network and using defineClass?

                  I apologize if I'm just missing something obvious here or misunderstanding the documentation. But I can't seem to completely reconcile what I understand you're telling me and the class loader documentation.
                  • 6. Re: Instantiation from serialized Class requires .class file in classpath?
                    EJP
                    Sure, but then what does the 2nd paragraph in the quote above mean?
                    It means that if you pass an array of bytes to ClassLoader.defineClass() you can define a class.
                    If you're saying ...
                    I'm not saying anything of the kind. I have absolutely no idea where you got that impression. Please read what I actually wrote.You need a ClassLoader to define a class. Serialization alone won't do it. defineClass() is a method of ClassLoader. Nothing to do with serialization.
                    • 7. Re: Instantiation from serialized Class requires .class file in classpath?
                      843790
                      If you're saying ...
                      I'm not saying anything of the kind. I have absolutely no idea where you got that impression. Please read what I actually wrote.You need a ClassLoader to define a class. Serialization alone won't do it. defineClass() is a method of ClassLoader. Nothing to do with serialization.
                      My apologies. I understand all that. I misinterpreted when you said:
                      Serialization only gives you objects for which you already have the required .class file loaded or loadable via the CLASSPATH.
                      ...and thought that you were responding to the defineClass example in the documentation, but you were clearly referring to serialization.

                      So just to summarize my understanding of the documentation...it is possible to send a chunk of bytes over the network, apply the class loader's defineClass() method to these bytes to define a class, and then instantiate an object with that class, all without having any .class files in your local classpath. Is that correct? (By "any .class files" I mean, of course, .class files relevant to the instantiated object and assuming that it is a single object and not an object graph requiring other .class files.)
                      • 8. Re: Instantiation from serialized Class requires .class file in classpath?
                        EJP
                        Obviously. That is what ClassLoader.defineClass() is for.

                        Usually you don't work at this level of detail: you would define a URLClassLoader, tell it where your .jar is in the network, and let it load the classes lazily as required.
                        • 9. Re: Instantiation from serialized Class requires .class file in classpath?
                          843790
                          Thanks for your help, ejp.
                          • 10. Re: Instantiation from serialized Class requires .class file in classpath?
                            843790
                            ejp wrote:
                            I would like to know if a Class object in isolation (i.e. a serialized Class object obtained over the network) contains all the information necessary to instantiate a new instance
                            No.
                            Would this answer change to "yes" if, along with the serialized object, the class definition was sent using:
                            protected void annotateClass(Class<?> cl); ==== stream ====> protected Class<?> resolvedClass(ObjectStreamClass desc); ?

                            I made a simple "proof of concept" program, that seems to work. Maybe a little practical testing might reveal this idea is a complete flop. A three-line class loader indicates to me that something is wrong. The annotateClass() method is hard-wired and maybe I can't fix that. Further, I am not using the ObjectStreamClass object. Some things are seriously out of place. Even so, the concept works. And so what does this say about serialization?
                            public class MyOIS extends ObjectInputStream {
                            
                                private MyClassLoader loader = new MyClassLoader();
                            
                                MyOIS() throws IOException {  super();   }
                                MyOIS(InputStream in) throws IOException { super(in); }
                            
                                protected Class<?> resolveClass(ObjectStreamClass desc) throws IOException, ClassNotFoundException {
                                    int bufSize = (int) super.readInt();
                                    byte[] buf = new byte[bufSize];
                                    this.read(buf);
                                    return loader.hello(buf, 0, buf.length);
                                }
                            
                                    public static class MyClassLoader extends ClassLoader {
                                        public Class<?> hello(byte[] image, int i, int j) {
                                            return defineClass(image, i, j);
                                        }
                                    }
                            }
                            The only reason I posted this hack is that maybe its a way to send an object's class along with the object, in the same stream? If not, then when are +"annotateClass()"+ and +"resolveClass()"+ used? I want to investigate distributed computing. Is this approach a dead-end? Completely wrong idea? Thanks.

                            Edited by: ichigoichi on Feb 5, 2010 1:56 PM
                            • 11. Re: Instantiation from serialized Class requires .class file in classpath?
                              EJP
                              I believe that may work but it's extremely inefficient to send the entire class along with every object. annotateClass() is usually only used to provide a URL where the class can be loaded if necessary.
                              • 12. Re: Instantiation from serialized Class requires .class file in classpath?
                                jschellSomeoneStoleMyAlias
                                skrappy wrote:
                                Since I have 1 server but many clients, for ease of use I would like to be able to drop a plug-in class file into my server's file system and have the server distribute the new class to all connected clients over the network using serialization. I would like to do this as simply as possible.
                                Reading this thread it seems that the real question is that you wish to deploy new classes to a client.

                                You might then wish to send instances as well but that is a secondary problem and one that you can solve once you have the class.

                                I believe there have been many solutions proposed for the first problem a googling for that, and not "serialization" (which is for instances not classes) would probably turn a number of solutions.
                                • 13. Re: Instantiation from serialized Class requires .class file in classpath?
                                  843790
                                  [placemat advertising business|http://www.aboutadeal.com/ads/a-practical-guide-to-restaurant-placemat-advertising-business]