1 Reply Latest reply: Jan 4, 2013 2:02 PM by jsmith RSS

    PixelWriter.setPixel() returns before completion/freezes application?

    898076
      Hi. I have an rgb BufferedImage, which I want to convert to rgb javafx's Image, but desaturated.

      The following works flawlessy, albeit without desaturation, of course:
      image = SwingFXUtils.toFXImage(frame.getBufferedImage(), null);
      Version with desaturation, which I tried:

      in constructor:
                  cachedBufferIn = new int[width*height*3];
                  cachedBufferOut = new byte[width*height*3];
      actual conversion, in a tight loop (60 frames per second):
                  image = cachedImage;
                  Raster reader = frame.getBufferedImage().getRaster();
                  reader.getPixels(0, 0, width, height, cachedBufferIn);
                  byte[] bufferOut = cachedBufferOut;
                  PixelWriter writer = image.getPixelWriter();
                  int length = width*height*3;
                  for(int index = 0; index < length; index += 3) {
                      byte v = (byte)((cachedBufferIn[index] +
                              (cachedBufferIn[index + 1] << 1) +
                                  cachedBufferIn[index + 2]) >> 2);
                      bufferOut[index + 0] = v;
                      bufferOut[index + 1] = v;
                      bufferOut[index + 2] = v;
                  }
                  writer.setPixels(0, 0, width, height, PixelFormat.getByteRgbInstance(),
                          bufferOut, 0, width*3);
      The code has two problems: (1) the output image sometimes looks truncated, as if setPixels() would occasionally return before completing the job fully, (2) also occasionally the whole application freezes, I do not know at which place, as in the debugger, it seems the freeze never happens. None of these two effects show up if the mentioned toFXImage() is used. Also, if BufferedImage is desaturated and then toFXImage() is used, the application works well too.

      I would like to ask you, what is wrong with the code, and how to make it fast? (the images come from a camera at 60fps).

      Edit: Has it something to do with fx2's thread? Perhaps not only event dispatching must be done there, but also certain (or all) image processing?
        • 1. Re: PixelWriter.setPixel() returns before completion/freezes application?
          jsmith
          Has it something to do with fx2's thread? Perhaps not only event dispatching must be done there, but also certain (or all) image processing?
          Seems likely - complete, executable source would be needed to verify.

          As you are mixing Swing and JavaFX code, you can do the processing of the Swing related objects on the Swing thread and the JavaFX related objects on the JavaFX thread.

          For example (assuming your image is a JavaFX image and you have been populating bufferOut from the Swing thread) wrap the update call to the image in a runLater so that the iamge is updated on the JavaFX thread =>
          Platform.runLater(new Runnable() {
            @Override public void run() {
              writer.setPixels(0, 0, width, height, PixelFormat.getByteRgbInstance(), bufferOut, 0, width*3);    
           }
          }
          Note that you may need to synchronize your runLater call to ensure it completes before refilling the buffer, as in: http://mail.openjdk.java.net/pipermail/openjfx-dev/2013-January/005239.html "Thread: invokeAndWait", but as Richard is quoted on that thread, be careful if you do so, so you can avoid deadlock. The synchronization may be necessary to ensure that the JavaFX thread has fully rendered the existing buffer before going back and trying to refill the buffer. An alternate solution to the synchronization would be to create a new buffer for each frame - which would generate an awful lot of garbage.

          The rule of thumb is, once a JavaFX object has been placed in an active Scene (e.g. a scene shown on a stage), you should only update the object on the JavaFX application thread. For example, if you go updating the pixels in an Image displayed in an ImageView from another thread, you may start updating the pixels at exactly the same time the image is being rendered, in which case you will get torn images where half the image is an old image and half is a new image. When you put everything back on JavaFX application thread, then the JavaFX system has the smarts to synchronize the updates and the rendering logic such that image tearing does not occur.

          When you were using SwingUtils.toFXimage you were passing in a null value for the image, which means that you are creating a completely new image each time you make that call rather than writing pixels inside an existing image, which may be why you were not experiencing an issue with it.