5 Replies Latest reply: Mar 4, 2013 2:37 PM by jsmith RSS

    Problem using byte indexed pixel format in setPixels method of PixelWriter

    Andipa
      I try to construct a byte array and set it to a WritableImage using PixelWriter's setPixels method.

      If I use an RGB pixel format, it works. If I use a byte-indexed pixel format, I get an NPE.
      The stride etc should be fine if I'm not mistaken.

      java.lang.NullPointerException
           at com.sun.javafx.image.impl.BaseByteToByteConverter.<init>(BaseByteToByteConverter.java:45)
           at com.sun.javafx.image.impl.General$ByteToByteGeneralConverter.<init>(General.java:69)
           at com.sun.javafx.image.impl.General.create(General.java:44)
           at com.sun.javafx.image.PixelUtils.getB2BConverter(PixelUtils.java:223)
           at com.sun.prism.Image$ByteAccess.setPixels(Image.java:770)
           at com.sun.prism.Image.setPixels(Image.java:606)
           at javafx.scene.image.WritableImage$2.setPixels(WritableImage.java:199)

      Short, self-contained example here:
      import java.nio.ByteBuffer;
      
      import javafx.application.Application;
      import javafx.scene.Scene;
      import javafx.scene.image.ImageView;
      import javafx.scene.image.PixelFormat;
      import javafx.scene.image.WritableImage;
      import javafx.scene.layout.BorderPane;
      import javafx.stage.Stage;
      
      public class IndexedColorTestApp extends Application {
      
          public static void main(String[] args) {
              launch(args);
          }
      
          @Override
          public void start(Stage primaryStage) {
              BorderPane borderPane = new BorderPane();
              Scene scene = new Scene(borderPane, 600, 1100);
              primaryStage.setScene(scene);
      
              ImageView imageView = new ImageView();
              borderPane.setCenter(imageView);
              primaryStage.show();
      
              int imageWidth = 200;
              int imageHeight = 200;
              WritableImage writableImage = new WritableImage(imageWidth, imageHeight);
      
              // this works
              byte[] rgbBytePixels = new byte[imageWidth * imageHeight * 3];
              PixelFormat<ByteBuffer> byteRgbFormat = PixelFormat.getByteRgbInstance();
              writableImage.getPixelWriter().setPixels(0, 0, imageWidth, imageHeight,
                                                       byteRgbFormat, rgbBytePixels, 0, imageWidth * 3);
              imageView.setImage(writableImage);
      
              // this throws an NPE in setPixels()
              byte[] indexedBytePixels = new byte[imageWidth * imageHeight];
              int[] colorPalette = new int[256];
              PixelFormat<ByteBuffer> byteIndexedFormat = PixelFormat.createByteIndexedInstance(colorPalette);
              writableImage.getPixelWriter().setPixels(0, 0, imageWidth, imageHeight,
                                                       byteIndexedFormat, indexedBytePixels, 0, imageWidth);
              imageView.setImage(writableImage);
          }
      
      }
      If there's no solution, maybe someone knows a workaround? We chose to use indexed format because of data size / performance reasons.

      Edited by: Andipa on 01.03.2013 10:52
        • 1. Re: Problem using byte indexed pixel format in setPixels method of PixelWriter
          jsmith
          You have found a platform bug, file it against the Runtime project at => http://javafx-jira.kenai.com with your sample code and a link back to this forum question.
          Byte indexed pixel formats seem like a feature which was never completely (or perhaps even hardly at all) implemented to me.

          The PixelFormat type your failed case is using is (PixelFormat.Type.BYTE_INDEXED):
          PixelFormat<ByteBuffer> byteIndexedFormat = PixelFormat.createByteIndexedInstance(colorPalette);
          System.out.println(byteIndexedFormat.getType());
          These are the valid PixelFormat types =>
          http://docs.oracle.com/javafx/2/api/javafx/scene/image/PixelFormat.Type.html
          BYTE_BGRA
          The pixels are stored in adjacent bytes with the non-premultiplied components stored in order of increasing index: blue, green, red, alpha.
          BYTE_BGRA_PRE
          The pixels are stored in adjacent bytes with the premultiplied components stored in order of increasing index: blue, green, red, alpha.
          BYTE_INDEXED
          The pixel colors are referenced by byte indices stored in the pixel array, with the byte interpreted as an unsigned index into a list of colors provided by the PixelFormat object.
          BYTE_RGB
          The opaque pixels are stored in adjacent bytes with the color components stored in order of increasing index: red, green, blue.
          INT_ARGB
          The pixels are stored in 32-bit integers with the non-premultiplied components stored in order, from MSb to LSb: alpha, red, green, blue.
          INT_ARGB_PRE
          The pixels are stored in 32-bit integers with the premultiplied components stored in order, from MSb to LSb: alpha, red, green, blue.
          As the native pixel format for a WritableImage is not the same as the pixel format you are using, the JavaFX platform needs to do a conversion by reading the pixels in one format and writing them in another format. To do this it must be able to determine a PixelGetter for your PixelFormat (the PixelGetter is an internal thing, not public API).

          And here is the source the determines the PixelGetter for a given PixelFormat type:
          http://hg.openjdk.java.net/openjfx/8/master/rt/file/06afa65a1aa3/javafx-ui-common/src/com/sun/javafx/image/PixelUtils.java
          119     public static <T extends Buffer> PixelGetter<T> getGetter(PixelFormat<T> pf) {
          120         switch (pf.getType()) {
          121             case BYTE_BGRA:
          122                 return (PixelGetter<T>) ByteBgra.getter;
          123             case BYTE_BGRA_PRE:
          124                 return (PixelGetter<T>) ByteBgraPre.getter;
          125             case INT_ARGB:
          126                 return (PixelGetter<T>) IntArgb.getter;
          127             case INT_ARGB_PRE:
          128                 return (PixelGetter<T>) IntArgbPre.getter;
          129             case BYTE_RGB:
          130                 return (PixelGetter<T>) ByteRgb.getter;
          131         }
          132         return null;
          133     }
          As you can see, the BYTE_INDEXED format is not handled and null is returned instead . . . this is the source of your NullPointerException.
          • 2. Re: Problem using byte indexed pixel format in setPixels method of PixelWriter
            jsmith
            Until the platform is fixed, the only way around this that I can see is to first convert your buffer to use a format which the pixel utilities will understand (by writing your own code and implementation to do the conversion), then pass that converted buffer through to the pixelwriter.
            • 3. Re: Problem using byte indexed pixel format in setPixels method of PixelWriter
              Andipa
              Thanks for the quick bug verification and suggestion for a workaround.

              Here's the bug I submitted: http://javafx-jira.kenai.com/browse/RT-28759
              • 4. Re: Problem using byte indexed pixel format in setPixels method of PixelWriter
                Andipa
                This triggers another question for me, I hope you read that one as well

                As you said
                Byte indexed pixel formats seem like a feature which was never completely (or perhaps even hardly at all) implemented to me.
                Does that mean byte-indexed images might ever be stored and processed as they are (like in Swing) or will they always need to be converted into RGB format internally in JavaFX because of an inherent framework restriction?
                • 5. Re: Problem using byte indexed pixel format in setPixels method of PixelWriter
                  jsmith
                  Does that mean byte-indexed images might ever be stored and processed as they are (like in Swing) or will they always need to be converted into RGB format internally in JavaFX because of an inherent framework restriction?
                  Internally, the framework can do whatever it wants in terms of storage and management of this data as long as it doesn't break the public API.

                  You will never have any guarantee that the framework will always continue to process the same data in the same way, only that the processing should have the same semantics - performance can (and likely will) vary wildly depending upon the internal implementation and hardware available.

                  JavaFX does map to multiple implementation pipelines, (e.g. software rendering, directX hardware rendering, opengl hardware rendering, special pipeline for low-end devices like a Raspberry Pi, etc). They may all handle this kind of stuff differently. For example, an iOS type rendering pipeline might end up using something weird like PVRTC at it's lowest implementation level: http://en.wikipedia.org/wiki/PVRTC. Also, JavaFX is a highlevel framework, so, to display a given image, multiple conversions may be done, for example. Your application's PixelFormat (e.g. byte indexed image) => to an internal PixelFormat (e.g. rgb image) => hardware renderable texture (e.g. PVRTC).

                  Obviously, this means that you don't have a lot of insight into conversions which may be going on within the underlying platform and the performance aspects of them unless somebody documents this internal stuff somewhere or you go through the open source code to see how the implementation works, or you benchmark your app with different formats.

                  As threads such as the following show, the correct pixel format choice and benchmarking can have an astounding impact on your application's performance if you are performing real-time updates to very large images:
                  Canvas performance "Canvas performance".

                  For now, byte-indexed images aren't even properly implemented in the underlying platform, so for your case it is a moot point.