1 2 Previous Next 18 Replies Latest reply: Jan 2, 2009 10:13 AM by 807589 RSS

    ImageIO.read() returns null?

    807589
      Right now I have 2 threads, one that constantly takes screenshots and sends it over to the other thread, and the other thread just displays them.
      I have a loop that takes a screenshot every 2 seconds, and stores it in a bufferedimage, then I send the image through

      ImageIO.write(image, "gif", oos);

      oos is an OutputStream.

      On the other side, I have

      currentImage=ImageIO.read(ois);

      where the ois is an InputStream.

      The problem I have is that currentImage is still null even after calling the read().
      Before the read method, I have the sender to send a byte of data to the receiver telling it that an image has been sent, and it can call the read(). However, this works for the first screenshot sent, and after the read() returns null, even though the write() has been called prior to the invocation of the read().


      The code for the sender is:
      for(int i =0;;i++){
                          image = capturer.createScreenCapture(rect);
                          temp = image;     
                          image = resizeImage(image,last,800,600);
                          last = temp;     
                          file = new File("screen"+i+".gif");
                          //System.out.println("writing! "+image.toString());
                          ImageIO.write(image, "gif", file);
                          ImageIO.write(image, "gif", oos);
                          System.out.println("written");
                          oss.write(77);
                          System.out.println("Written signal byte");
                          for(;;){
                               System.out.println("waiting for server to read..");
                               int seeker = (int)ois.read();     
                               if(seeker==77){
                                    System.out.println("Data has been read");
                                    oos.flush();
                                    break;
                               }
                               if(seeker==88&&image!=null){
                                    System.out.println("Sending again");
                                    oos.flush();
                                    ImageIO.write(image, "gif", oos);
                               }
                          }     
                          System.out.println("received image confirmation");
                          if(exit){
                               break;
                          }
                     }
      And the code for the receiver is:
      while(true){
                               BufferedImage currentImage;
                                    int seeker = sig.read();//waiting for the sender to send an image
                               System.out.println("Ready to read!");
                               currentImage=ImageIO.read(ois);
                               if(currentImage==null){
                                    for(;;){
                                         System.out.println("received nothing.. requesting again");
                                         oos.write(88);//teling the server that nothing has been read
                                         currentImage=ImageIO.read(ois);          
                                         if(currentImage!=null)
                                              break;
                                         Thread.sleep(500);
                                    }
                               }
                               System.out.println("Message Received: " +currentImage.toString());
                               oos.write(77);//telling the sender that the image has been received
                               oos.flush();
                               File f = new File("1.jpg");
                               if(currentImage!=null){
                                    ImageIO.write(currentImage, "gif", f);
                                    r.update(currentImage);
                               }
                               boolean exit=false;
                               if(exit){
                                    break;
                               }
                          }
        • 1. Re: ImageIO.read() returns null?
          807589
          I've found that apparently if I write it twice, like:
          ImageIO.write(image, "gif", oos);
               ImageIO.write(image, "gif", oos);
          Then on the other side, I read it twice. like:
          currentImage=ImageIO.read(ois);
          System.out.println(currentImage.toString());
          currentImage=ImageIO.read(ois);
          System.out.println(currentImage.toString());
          The first time currentImage will be the image sent. The second time I will get a null pointer.
          However, If I read using ois.read();, I don't get -1, therefore there's still data to be read.

          Can anyone help me?
          • 2. Re: ImageIO.read() returns null?
            EJP
            Your code doesn't cope cprrectly with the case where the signal byte read isn't 77 or 88. You just keep reading until it is and you don't even log the error anywhere. So you are quite possibly losing synchronization between writer and reader. So the image won't be where you think it is in the stream. So it won't be understood. So no image decoder can be found for it. So ImageIO.read() returns null just like it says it will in the Javadoc.
            • 3. Re: ImageIO.read() returns null?
              807589
              Hmm, any idea on how I can ensure that the reader/writer are synchronized?
              • 4. Re: ImageIO.read() returns null?
                807589
                And, even if from my second post, it seems that the image is fine the first time, but gets messed up the second time, any ideas why?
                • 5. Re: ImageIO.read() returns null?
                  EJP
                  Hmm, any idea on how I can ensure that the reader/writer are synchronized?
                  Hmm, with the 'synchronized' keyword?
                  • 6. Re: ImageIO.read() returns null?
                    807589
                    I've been exploring this issue and this seems to be a more fundamental problem then expected.

                    I'll use JPEG as an example. If an image is a jpeg image then the first two bytes are 0xFF and 0xD8. Now suppose I have a server/client relationship. The ImageIO.writes(...) on the server side occur perfectly. For example, if I write two jpeg images on the server side, and then read the bytes directly from the input stream on the client side then I can find exactly two JPEG images (as indicated by the 0xFF and 0xD8 bytes). Once I start using ImageReaders, though, it breaks down.

                    For example, on the client side the moment I do this
                    InputStream socketStream = socket.getInputStream();
                    ImageInputStream inputStream = ImageIO.createImageInputStream(socketStream);
                    ImageReader r = ImageIO.getImageReaders(inputStream).next()
                    then I can now only detect one jpeg image in the socket stream. This, however, is reasonable since the ImageIO class has to read some bytes from the InputStream to determine the correct ImageReader. Once I read the image though with the following code
                    r.setInput(inputStream,true,false);
                    r.read(0);
                    then I am no longer able to detect the 2nd JPEG in the socket stream. This means the reader decided to read some bytes of the 2nd JPEG (and thus consumed the 0xFF and 0xD8 signature bytes). Astonishingly though, looking deeper into it - it turned out that there were no bytes left in the socket input stream. The JPEGReader consumed the entire+ stream in reading the first JPEG image. Low and behold
                    r.read(1);
                    gave me the second image. If I had instead decided to use ImageIO.read(..) two consecutive times then the second time would return a null, since the first read consumes the whole stream. I think JPEGImageReader decides to this because it uses a native method for decoding the data.

                    Using a single reader (in JPEG's case) to read the continuous flow of images is not a viable option (even though it works). This is because ImageInputStream caches the data (out of necessity), and you'll end up with an out of memory error in due time. And in fact, because JPEGImageReader tries to consume the whole stream, the read command will actualy block until the stream is closed on the server side. Basically you can't really stream JPEG images by this method.

                    Now how about GIF images? Well, fortunately the GIFImageReader dosen't try to consume the whole stream, so it won't block. However, it seems like it does consume a few bytes more then the actual image data . This might have something to do with the GIF specification. When it reads the junk data from the 2nd image at some point it decides "ok I can't get anymore stuff" and returns the BufferedImage. But because it read some data from the second image, the header information identifying the second image is now screwed up. So ImageIO ends up thinking it can't decode the format of the second image and ends up returning null (with bytes still left in the input stream).

                    It seems like PNG is the same as GIF. I'll explore some more, but for now I point you to MichaelDGagnon's post (#6) in this thread

                    [http://forums.sun.com/thread.jspa?threadID=5279970&tstart=15|http://forums.sun.com/thread.jspa?threadID=5279970&tstart=15]

                    where he/she uses serialization for transporting encoded images over a connection.
                    • 7. Re: ImageIO.read() returns null?
                      807589
                      Right now I have a temporary solution; I just close the socket then reconnect for every loop. This seems to work perfectly, except I don't think it's the most efficient way since I think it's pretty laggy.
                      • 8. Re: ImageIO.read() returns null?
                        807589
                        HappyCamper5 wrote:
                        Right now I have a temporary solution; I just close the socket then reconnect for every loop. This seems to work perfectly, except I don't think it's the most efficient way since I think it's pretty laggy.
                        Yes this would work given what I've found out. And yes, it's definitely not optimal. An alternative would be to

                        1) On server side read the image into byte array (using ByteArrayOutputStream). Prior to sending this this byte array send an int that tells the client the length of the byte array about to be sent.

                        2) On client side read the int describing the size of the byte array. Read from the socket stream the given length of bytes. Create a ByteArrayInputStream of the bytes and decode this stream using ImageIO. Discard the ByteArrayInputStream, read the next int from the socket stream, and repeat.

                        In this way the ImageReader dosen't even have a chance to consume more bytes then it should. If the server keeps sending the size of the image data then the client can keep reading only that length of bytes.
                        • 9. Re: ImageIO.read() returns null?
                          807589
                          The problem with that is, with a large image, the speed of transfer is going to be really slow, whereas gif has a built in compression, which makes the speed of transfer faster.
                          • 10. Re: ImageIO.read() returns null?
                            807589
                            What I'm saying is.
                            ImageIO.write(image, "gif", oos);
                            is pretty much the same as
                            ByteArrayOutputStream ouput = ByteArrayOutputStream();
                            ImageIO.write(image,"gif",output);
                            oos.write(output.toByteArray());
                            With the second case, though, you can send the size of the byte array first.
                            • 11. Re: ImageIO.read() returns null?
                              807589
                              Ohhh Thanks alot, that worked. But I've encountered another problem. After running this program long enough to take about 30-40 screenshots, it gives me the:
                              Exception in thread "Thread-1" java.lang.OutOfMemoryError: Java heap space
                                   at sSocket.run(sSocket.java:84)
                              Line 84 instantiates a byte array that is used to store the data read.
                              • 12. Re: ImageIO.read() returns null?
                                807589
                                Hm... that's tricky. You might not be calling flush on the server side output stream after you write an image. This is an important thing to do since the stream might be buffering until an OutOfMemoryError. If you look at the code, the ImageIO class calls flush after every write operation. It's either this or the server side is writing images faster then the client can read them.

                                On server side
                                while(/*Still connected*/) {
                                   ...
                                
                                    ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
                                    ImageIO.write(image,"gif",outputStream);
                                    //unless you do something else with the image let it be available for
                                    //garbage collection now
                                    image = null; 
                                
                                    int length = outputStream.size();
                                    byte[] size = new byte[]{(byte) ((length>>24)&0xff),
                                                             (byte) ((length>>16)&0xff),
                                                             (byte) ((length>> 8)&0xff),
                                                             (byte) ((length    )&0xff)};
                                    oos.write(size);
                                
                                    //I had this statement backwards. This is more memory efficient
                                    outputStream.writeTo(oos);
                                
                                    outputStream.flush(); //I'm not sure if this is important
                                    oos.flush(); //this is important
                                    outputStream == null;
                                
                                    ...
                                  }
                                On client side
                                byte[] size = new byte[4];
                                while (true) {
                                    try {
                                        while (ois.read(size) == -1) {
                                            try{Thread.sleep(50);} //wait a bit for some input
                                            catch(InterruptedException e) {
                                                System.err.println(e.getMessage());
                                            }
                                        }
                                
                                        int length = ((size[0]&0xff)<<24) |
                                                     ((size[1]&0xff)<<16) |
                                                     ((size[2]&0xff)<< 8) | 
                                                      (size[3]&0xff);
                                
                                        byte[] imageData = new byte[length];
                                        ois.read(imageData);
                                        ByteArrayInputStream inputStream = 
                                                new ByteArrayInputStream(imageData);
                                        BufferedImage theImage = ImageIO.read(inputStream);
                                        inputStream = null;
                                        imageData = null;
                                
                                        //do stuff with the image
                                    } catch (java.io.IOException e) {
                                        //stream has been closed on server side
                                        break;
                                    }
                                }
                                Edited by: Maxideon on Jan 1, 2009 9:38 PM
                                One second. The code is slightly faulty. I'm getting intermittent NegativeArraySizeExceptions. The client side code is wrong. I'll fix it when I figure it out.
                                • 13. Re: ImageIO.read() returns null?
                                  EJP
                                  byte[] size = new byte[]{(byte) ((length>>24)&0xff),
                                  (byte) ((length>>16)&0xff),
                                  (byte) ((length>> 8)&0xff),
                                  (byte) ((length )&0xff)};
                                  Those shifts should all be >>>.
                                  while (ois.read(size) == -1) {
                                  try{Thread.sleep(50);} //wait a bit for some input
                                  That doesn't do any such thing. What it does is read from the input stream, blocking until it does so. The subsequent read() method will block so sleeping here is completely pointless.
                                  int length = ((size[0]&0xff)<<24) |
                                  ((size[1]&0xff)<<16) |
                                  ((size[2]&0xff)<< 8) |
                                  (size[3]&0xff);
                                  Here you are assuming you read four bytes, because you threw away the result of the read. Unsafe code.
                                  byte[] imageData = new byte[length];
                                  ois.read(imageData);
                                  Here you are throwing away the result of the read again, and assuming you filled the array. This is not very likely over a network.

                                  I would use DataInputStream.readInt()/readFully for these operations, with the corresponding DataOutputStream.writeInt() at the sender.
                                  • 14. Re: ImageIO.read() returns null?
                                    807589
                                    The bitshift operators are correct. You're correct about the sleeping thing. I don't know why I did that. And the negative array size exception was caused by me assuming I completely read the image data in one go. The fix is trivial.

                                    Client side
                                    while (true) {
                                        try {
                                            byte[] size = new byte[4];
                                            int read = 0;
                                            while(read < 4)
                                                read += ois.read(size,read,4-read);
                                           
                                            int length = ((size[0]&0xff)<<24) |
                                                         ((size[1]&0xff)<<16) |
                                                         ((size[2]&0xff)<< 8) | 
                                                          (size[3]&0xff);
                                     
                                            byte[] imageData = new byte[length];
                                            
                                            read = 0;
                                            while(read < length)
                                                read += ois.read(imageData, read, length-read);
                                            
                                            ByteArrayInputStream inputStream = 
                                                    new ByteArrayInputStream(imageData);
                                            BufferedImage theImage = ImageIO.read(inputStream);
                                            inputStream = null;
                                            imageData = null;
                                     
                                            //do stuff with the image
                                        } catch (java.io.IOException e) {
                                            //stream has been closed on server side
                                            break;
                                        }
                                    }
                                    1 2 Previous Next