12 Replies Latest reply on Apr 30, 2009 3:40 AM by 796280

    Problem with ObjectInputStream/ObjectOutputStream

    796280
      Hello all,

      I've been researching, poking, prodding, and testing all day and I haven't any luck with this.

      Quick note: I'm using Java 1.4.2 for this project, running and debugging in Eclipse 3.3.2.
      To prevent flames, I must first state that THIS IS NOT MY CODE--I'm maintaining it.
      Some class names have been altered to protect proprietary information.

      For those who don't want to read the whole post (with details about my tests), here's a summary: I'm getting intermittent success with reading/writing objects from/to a file.

      Here's where my data is loaded from a file.
        private void loadOption() {
          FileInputStream f;
          ObjectInput s = null;
          try {
            f = new FileInputStream("./config/foobar.opt");
            s = new ObjectInputStream(f);
            while (true) {
              FooBar fb = (FooBar) s.readObject();
              Root.fooBars.add(fb); // Root.fooBars is a LinkedList
            }
          }
          catch (Exception ex) {
               ex.printStackTrace();
          }
          try {
            s.close();
          }
          catch (Exception ex) {}
      
        }
      Here's the part of the application that writes my data to a file (:
      class saveThread extends Thread{
        public boolean bComplete = false;
        public void run(){
          try {
          FileOutputStream f = new FileOutputStream("./config/foobar.opt");
          ObjectOutput s = new ObjectOutputStream(f);
          for (int i = 0; i < Root.fooBars.size(); i++) { // again, Root.fooBars is a LinkedList
            s.writeObject(Root.fooBars.get(i));
          }
      
          s.flush();
          f.close();
          bComplete = true;
        }
        catch (Exception ex) {
          bComplete = true;
        }
        }
      
        public boolean getStatus(){
          return bComplete;
        }
      };
      which is called elsewhere like so:
          saveThread threadObj = new saveThread();
          threadObj.start();
      The FooBar class a whole ****load of different member variable types:
      int, String, boolean, List, and ImageSerializable (a custom class--implements Serializable--containing two member vars: A String and a BufferedImage).

      Here is my workflow to reproduce the problem:
      1) Delete foobar.opt
      2) Create new data and write to foobar.opt (by running a saveThread)
      3) Close and restart application
      4) On application startup, loadOption() is called and a exception is thrown on this line:
      FooBar fb = (FooBar) s.readObject();
      Here's the exception stack trace:
      java.io.StreamCorruptedException: invalid type code: 78
           at java.io.ObjectInputStream.readClassDesc(Unknown Source)
           at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
           at java.io.ObjectInputStream.readObject0(Unknown Source)
           at java.io.ObjectInputStream.defaultReadFields(Unknown Source)
           at java.io.ObjectInputStream.readSerialData(Unknown Source)
           at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
           at java.io.ObjectInputStream.readObject0(Unknown Source)
           at java.io.ObjectInputStream.readObject(Unknown Source)
           at java.util.LinkedList.readObject(Unknown Source)
           at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
           at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
           at java.lang.reflect.Method.invoke(Unknown Source)
           at java.io.ObjectStreamClass.invokeReadObject(Unknown Source)
           at java.io.ObjectInputStream.readSerialData(Unknown Source)
           at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
           at java.io.ObjectInputStream.readObject0(Unknown Source)
           at java.io.ObjectInputStream.defaultReadFields(Unknown Source)
           at java.io.ObjectInputStream.readSerialData(Unknown Source)
           at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
           at java.io.ObjectInputStream.readObject0(Unknown Source)
           at java.io.ObjectInputStream.readObject(Unknown Source)
           at java.util.ArrayList.readObject(Unknown Source)
           at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
           at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
           at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
           at java.lang.reflect.Method.invoke(Unknown Source)
           at java.io.ObjectStreamClass.invokeReadObject(Unknown Source)
           at java.io.ObjectInputStream.readSerialData(Unknown Source)
           at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
           at java.io.ObjectInputStream.readObject0(Unknown Source)
           at java.io.ObjectInputStream.defaultReadFields(Unknown Source)
           at java.io.ObjectInputStream.readSerialData(Unknown Source)
           at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
           at java.io.ObjectInputStream.readObject0(Unknown Source)
           at java.io.ObjectInputStream.readObject(Unknown Source)
        • 1. Re: Problem with ObjectInputStream/ObjectOutputStream
          796280
          If I repeat the exact same test (identical data and steps), I now get this exception on the same line of code:
          ava.lang.IllegalStateException: unread block data
               at java.io.ObjectInputStream$BlockDataInputStream.setBlockDataMode(Unknown Source)
               at java.io.ObjectInputStream.readObject0(Unknown Source)
               at java.io.ObjectInputStream.defaultReadFields(Unknown Source)
               at java.io.ObjectInputStream.readSerialData(Unknown Source)
               at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
               at java.io.ObjectInputStream.readObject0(Unknown Source)
               at java.io.ObjectInputStream.readObject(Unknown Source)
          Performing an identical test a 3rd time (as I'm drafting this post) yields this exception on the same line of code:
          java.io.EOFException
               at java.io.ObjectInputStream$BlockDataInputStream.peekByte(Unknown Source)
               at java.io.ObjectInputStream.readObject0(Unknown Source)
               at java.io.ObjectInputStream.readObject(Unknown Source)
          The difference is, given the way loadOption() is written, I expect an EOFException if everything runs smoothly.... Which it does.

          After this successful load, if I run the saveThread again, close the app and restart, loadOption will give me this exception, which refers to some other classes in the application:
          java.lang.ClassCastException: cannot assign instance of java.lang.String to field root.Bitdef.parent of type root.TObject in instance of root.Bitdef
               at java.io.ObjectStreamClass$FieldReflector.setObjFieldValues(Unknown Source)
               at java.io.ObjectStreamClass.setObjFieldValues(Unknown Source)
               at java.io.ObjectInputStream.defaultReadFields(Unknown Source)
               at java.io.ObjectInputStream.readSerialData(Unknown Source)
               at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
               at java.io.ObjectInputStream.readObject0(Unknown Source)
               at java.io.ObjectInputStream.readObject(Unknown Source)
               at java.util.LinkedList.readObject(Unknown Source)
               at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
               at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
               at java.lang.reflect.Method.invoke(Unknown Source)
               at java.io.ObjectStreamClass.invokeReadObject(Unknown Source)
               at java.io.ObjectInputStream.readSerialData(Unknown Source)
               at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
               at java.io.ObjectInputStream.readObject0(Unknown Source)
               at java.io.ObjectInputStream.defaultReadFields(Unknown Source)
               at java.io.ObjectInputStream.readSerialData(Unknown Source)
               at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
               at java.io.ObjectInputStream.readObject0(Unknown Source)
               at java.io.ObjectInputStream.readObject(Unknown Source)
               at java.util.LinkedList.readObject(Unknown Source)
               at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
               at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
               at java.lang.reflect.Method.invoke(Unknown Source)
               at java.io.ObjectStreamClass.invokeReadObject(Unknown Source)
               at java.io.ObjectInputStream.readSerialData(Unknown Source)
               at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
               at java.io.ObjectInputStream.readObject0(Unknown Source)
               at java.io.ObjectInputStream.defaultReadFields(Unknown Source)
               at java.io.ObjectInputStream.readSerialData(Unknown Source)
               at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
               at java.io.ObjectInputStream.readObject0(Unknown Source)
               at java.io.ObjectInputStream.readObject(Unknown Source)
               at java.util.ArrayList.readObject(Unknown Source)
               at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
               at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
               at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
               at java.lang.reflect.Method.invoke(Unknown Source)
               at java.io.ObjectStreamClass.invokeReadObject(Unknown Source)
               at java.io.ObjectInputStream.readSerialData(Unknown Source)
               at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
               at java.io.ObjectInputStream.readObject0(Unknown Source)
               at java.io.ObjectInputStream.defaultReadFields(Unknown Source)
               at java.io.ObjectInputStream.readSerialData(Unknown Source)
               at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
               at java.io.ObjectInputStream.readObject0(Unknown Source)
               at java.io.ObjectInputStream.readObject(Unknown Source)
          Any thoughts on what might be wrong? Is the original author's approach here completely FUBAR'ed? I'm pretty sure that this functionality worked before the FooBar class included the custom ImageSerializable objects.

          I'm pretty lost on this.

          Thanks,
          -Thok
          • 2. Re: Problem with ObjectInputStream/ObjectOutputStream
            EJP
            catch (Exception exc) {
            catch (EOFException exc) {
            // EOFException is 'expected', not an error, no need to show it.
            } catch (Exception ex) {
            s.close();
            s.flush();
            f.close();
            s.close();
            bComplete = true;
            }
            catch (Exception ex) {
            bComplete = true;
            Surely that should be 'false'! And the exception should be logged, not ignored. Anything could have happened! Actually I would say if you get an IOException here you should delete the output file and notify the application somehow.

            I don't see any reason for this to be a separate Thread so I would change this method to throw IOException and adjust the calling method appropriately.
            java.io.StreamCorruptedException: invalid type code: 78
            Fix the errors above and try again.

            Does the added class have custom readObject(), writeObject(), readResolve(), writeReplace() methods?
            • 3. Re: Problem with ObjectInputStream/ObjectOutputStream
              796280
              ejp wrote:
              catch (Exception exc) {
              catch (EOFException exc) {
              // EOFException is 'expected', not an error, no need to show it.
              } catch (Exception ex) {
              s.close();
              s.flush();
              f.close();
              s.close();
              bComplete = true;
              }
              catch (Exception ex) {
              bComplete = true;
              Surely that should be 'false'! And the exception should be logged, not ignored. Anything could have happened! Actually I would say if you get an IOException here you should delete the output file and notify the application somehow.
              I agree. I run into WTFs like this on a regular basis with this project. =P
              I don't see any reason for this to be a separate Thread so I would change this method to throw IOException and adjust the calling method appropriately.
              I'm wondering that too, but then again, with a relatively small data set to write to the file, it still ends up being ~3mb. They previously did not have the save function in a seperate thread but with the addition of the images, it might make more sense. Maybe I'll just have to tweak the threading a bit to make it behave.
              java.io.StreamCorruptedException: invalid type code: 78
              Fix the errors above and try again.

              Does the added class have custom readObject(), writeObject(), readResolve(), writeReplace() methods?
              The custom class (FooBar) does not, but I mentioned that FooBar has some member vars of type ImageSerializable; ImageSerializable does have custom writeObject() and readObject() methods.

              Which reminds me: my experience with object serialization is pretty much nil--should the FooBar class and ImageSerializable class include a serialVersionUID? Or is this just optional?

              Thanks for your help.
              -Thok
              • 4. Re: Problem with ObjectInputStream/ObjectOutputStream
                796280
                It looks like I fixed the error. I implemented the readObject() and writeObject() methods from the Serializable interface and explicitly read in/wrote out each member variable that I need for the FooBar class.

                I have to do more stress testing, but it appears everything is working great.

                Thanks,
                -Thok
                • 5. Re: Problem with ObjectInputStream/ObjectOutputStream
                  EJP
                  That shouldn't make any difference! I would be suspicious of the custom methods in the image class. Can you post them here?
                  • 6. Re: Problem with ObjectInputStream/ObjectOutputStream
                    796280
                    ejp wrote:
                    That shouldn't make any difference! I would be suspicious of the custom methods in the image class. Can you post them here?
                    Sure:
                    // read/write object methods for ImageSerializable:
                    // name is a String member var
                     private void writeObject(java.io.ObjectOutputStream out)throws IOException{
                        out.writeObject(name);
                        ImageIO.write(image,"jpeg",ImageIO.createImageOutputStream(out));
                      }
                    
                     private void readObject(java.io.ObjectInputStream in)throws IOException, ClassNotFoundException{
                        name=(String)in.readObject();
                        image=ImageIO.read(ImageIO.createImageInputStream(in));
                      }
                    The other change I made was in the saveThread (added synchronization):
                    class saveThread extends Thread{
                      public boolean bComplete = false;
                      public void run(){
                        save();
                      }
                      
                      private synchronized void save() {
                           ObjectOutput s = null;
                           try {
                                FileOutputStream f = new FileOutputStream("./config/foobar.opt");
                                s = new ObjectOutputStream(f);
                                for (int i = 0; i < Root.fooBars.size(); i++) {
                                     s.writeObject(Root.fooBars.get(i));
                                }
                                s.flush();
                                f.close();
                                bComplete = true;
                           }
                    This appeared to have no effect, but I was informed by another developer that it should be synchronized 'just in case'. I have had little or no exposure to threading so I'll have to take his word for it.

                    The addition of the custom readObject() and writeObject() methods to the FooBar class consistently gave me positive results--I'll take what I can get. =P
                    • 7. Re: Problem with ObjectInputStream/ObjectOutputStream
                      EJP
                      The writeObject() method should start with out.defaultWriteObject(), and the readObject() method should start with in.defaultReadObject(). (Always, in any class). The 'image' member should be declared as transient. All the non-transient members, presumably including 'name', will be written and read by the defaultXXX() methods so the writes and reads for 'name' can probably be deleted.+All+ your serializable classes should have serialVersionUIDs.
                      • 8. Re: Problem with ObjectInputStream/ObjectOutputStream
                        796280
                        ejp wrote:
                        The writeObject() method should start with out.defaultWriteObject(), and the readObject() method should start with in.defaultReadObject(). (Always, in any class). The 'image' member should be declared as transient. All the non-transient members, presumably including 'name', will be written and read by the defaultXXX() methods so the writes and reads for 'name' can probably be deleted.+All+ your serializable classes should have serialVersionUIDs.
                        It appears that both implementations work, but your approach is cleaner and easier (and noob-less).

                        I've updated the custom writeObject/readObject methods for FooBar and ImageSerializable classes per your suggestion and added a serialVersionUID to each as well.

                        Again, you have my thanks.
                        -Thok

                        Edited by: Thok on Apr 29, 2009 4:36 PM
                        • 9. Re: Problem with ObjectInputStream/ObjectOutputStream
                          EJP
                          Great, now note that if you're going to persist these streams anywhere you cannot change these methods again, unless you also change the serialVersionUID to indicate that serialization-compatibility is broken. However you can add members to the classes as per the rules for Versioning in the Serialization specification.

                          I would have thought you could now delete the custom methods in FooBar, have you tried that? If they only contain defaultXXXObject() calls they aren't doing anything that wouldn't happen anyway.
                          • 10. Re: Problem with ObjectInputStream/ObjectOutputStream
                            796280
                            ejp wrote:
                            Great, now note that if you're going to persist these streams anywhere you cannot change these methods again, unless you also change the serialVersionUID to indicate that serialization-compatibility is broken. However you can add members to the classes as per the rules for Versioning in the Serialization specification.
                            Hopefully I won't have to mess with this; I don't think we'll be making too many changes to this code. Regardless, that's good to know.
                            I would have thought you could now delete the custom methods in FooBar, have you tried that? If they only contain defaultXXXObject() calls they aren't doing anything that wouldn't happen anyway.
                            I guess I'm a little confused. Let me see if I have this is right:
                            In the FooBar class, I have a bunch of member vars (String, int, and List types). Then I have some vars of the ImageSerializable type which are declared transient.
                            Here's my writeObject() method for FooBar:
                            private void writeObject (java.io.ObjectOutputStream out) throws IOException {
                                    out.defaultWriteObject();
                                    out.writeObject(this.image1); // <- ImageSerializable vars
                                    out.writeObject(this.image2);
                                    out.writeObject(this.image3);
                                    out.writeObject(this.image4);
                               }
                            And readObject():
                            private void readObject (java.io.ObjectInputStream in) throws IOException,ClassNotFoundException {
                                    in.defaultReadObject();
                                    this.image1= (ImageSerializable)in.readObject(); // <- ImageSerializable vars
                                    this.image2= (ImageSerializable)in.readObject();
                                    this.image3= (ImageSerializable)in.readObject();
                                    this.image4= (ImageSerializable)in.readObject();
                               }
                            In the ImageSerializable class, I have two member vars: a String and a BufferedImage. The BufferedImage is transient.
                            My read and write methods are as follows:
                            private void writeObject(java.io.ObjectOutputStream out)throws IOException{
                                 out.defaultWriteObject();
                                ImageIO.write(image,"jpeg",ImageIO.createImageOutputStream(out));
                              }
                            
                            private void readObject(java.io.ObjectInputStream in)throws IOException, ClassNotFoundException{
                                 in.defaultReadObject();
                                image=ImageIO.read(ImageIO.createImageInputStream(in));
                              }
                            Is this sane or am I just stabbing in the dark? Having these methods appears to be necessary, because if I do not use custom read/write methods in FooBar, I get intermittent failures (mostly OptionalDataExceptions and StreamCorruptedExceptions when reading in the object). If I do it this way, it works consistently and hasn't failed yet.

                            The OptionalDataException is thrown (consistently) after 1 ImageSerializable object is read in (the first of 4). Here are some debugging statements and exception output which encouraged me to override my read/write methods (as opposed to leaving it all up to Java to figure out--which causes failures).
                            img srlzbl: readObject
                            (success)
                            img srlzbl: readObject
                            eof: true
                            length: 0
                            java.io.OptionalDataException
                                 at java.io.ObjectInputStream.readObject0(Unknown Source)
                                 at java.io.ObjectInputStream.readObject(Unknown Source)
                                 at java.util.LinkedList.readObject(Unknown Source)
                                 at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
                                 at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
                                 at java.lang.reflect.Method.invoke(Unknown Source)
                                 at java.io.ObjectStreamClass.invokeReadObject(Unknown Source)
                                 at java.io.ObjectInputStream.readSerialData(Unknown Source)
                                 at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
                                 at java.io.ObjectInputStream.readObject0(Unknown Source)
                                 at java.io.ObjectInputStream.defaultReadFields(Unknown Source)
                                 at java.io.ObjectInputStream.readSerialData(Unknown Source)
                                 at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
                                 at java.io.ObjectInputStream.readObject0(Unknown Source)
                                 at java.io.ObjectInputStream.readObject(Unknown Source)
                            ... and so on
                            It is entirely possible that my solution works, just that it may not have anything to do directly with actual root cause of the problem. I hope that's not the case but I am new to this stuff, so who knows.

                            -Thok
                            • 11. Re: Problem with ObjectInputStream/ObjectOutputStream
                              EJP
                              Here's my writeObject() method for FooBar:
                              And readObject():
                              You need those if & only if the image members are transient. If you make the image members non-transient and delete both methods, it should should have more or less exactly the same effect (although the streams will be slightly different).
                              In the ImageSerializable class, I have two member vars: a String and a BufferedImage. The BufferedImage is transient.
                              My read and write methods are as follows:
                              This is all OK.
                              Is this sane or am I just stabbing in the dark? Having these methods appears to be necessary, because if I do not use custom read/write methods in FooBar, I get intermittent failures (mostly OptionalDataExceptions and StreamCorruptedExceptions when reading in the object). If I do it this way, it works consistently and hasn't failed yet.
                              Well in that case I suppose you'd better leave it alone, but really it should work with no transients and no serialization methods in FooBar. The tricky stuff is really only in the ImageSerializable class.
                              • 12. Re: Problem with ObjectInputStream/ObjectOutputStream
                                796280
                                Cool. Well, I guess we'll call the bug fixed.

                                Thanks again for your help. It is much appreciated.

                                Cheers,
                                -Thok