This discussion is archived
4 Replies Latest reply: Mar 19, 2013 3:12 PM by jschellSomeoneStoleMyAlias RSS

OutofMemory while loading/writing huge image

990282 Newbie
Currently Being Moderated
Hello Experts,
Ii am in very specific problem. i need to load in my java program an image file having size of 18000 * 18000 in pixels so for that i have to use BufferedImage (which creates whole file in a memory at a time) so for that i need (18000 * 18000 * 4 )/ (1024*1024) = 1030 MB of memory approximately , so probably it throws and OutofMemoryException normally. currently i am using 32 bit machine and i have 4 MB of RAM. so if i run this code on 64 bit machine with Xmx 2048 mb would i succeed to load the image without out of memory error ?

if i want to run this code in 32 bit machine with 4GB of Physical Ram what should i do ?

if JVM allows me to allocate such a heap space is there any chance by above specification we exceed the maximum array size limit ?

the following code line 5 throws and exception

public DataBufferByte(int size, int numBanks) {
super(TYPE_BYTE, size, numBanks);
bankdata = new byte[numBanks][];
for (int i= 0; i < numBanks; i++) {
bankdata[i] = new byte[size];
}
data = bankdata[0];
}

Thank You
Mihir Parekh
  • 1. Re: OutofMemory while loading/writing huge image
    gimbal2 Guru
    Currently Being Moderated
    You match the hardware to the software, not the software to the hardware. If you want to process huge data sets do that on a machine capable of actually loading those. Hardware is cheap, you are expensive.

    But yeah if you must

    1) does it have to be 32 bits? Can it for example be an 8 bit image? That cuts the size down considerably. You won't be able to use ImageIO to load the image as that will create the huge bufferedimage again, you'll need some other third party (and possibly native) API to be able to stream the data in.

    2) don't work with one huge image, make sure the image is cut up into smaller pieces before your application gets them

    Whatever you do, the hardware upgrade is the easy fix, the software fix is going to be hard and you won't find something in the standard JDK to do it. I make that statement so that someone else will now be motivated to prove me wrong.
  • 2. Re: OutofMemory while loading/writing huge image
    990282 Newbie
    Currently Being Moderated
    Hello gimbal2

    thank you for your prompt reply , please see the following code,

    protected void transcode(Document document, String uri,
    TranscoderOutput output) throws TranscoderException {

    // Sets up root, curTxf & curAoi
    super.transcode(document, uri, output);

    Filter f = this.root.getGraphicsNodeRable(true);

    RenderContext rc = new RenderContext(curTxf, null, null);
    RenderedImage img = f.createRendering(rc);

    // prepare the image to be painted
    int w = img.getWidth();
    int h = img.getHeight();

    try {
    int bands = img.getSampleModel().getNumBands();
    int[] off = new int[bands];
    for (int i = 0; i < bands; i++)
    off[i] = i;
    SampleModel sm = new PixelInterleavedSampleModel(
    DataBuffer.TYPE_BYTE, w, (100000 + w - 1) / w, bands, w
    * bands, off);

    RenderedImage rimg = new FormatRed(GraphicsUtil.wrap(img), sm);

    PngEncoder enc = new PngImageEncoder(output
    *.getOutputStream(), null);*
    enc.encode(rimg);
    } catch (IOException ioe) {
    ioe.printStackTrace();
    }
    }

    the line in bold above throws an exception which is because of BufferedImage buffer .

    as per your second suggestion how can i cut RenderedImage into small chunks ?

    if i get success in it then i can write each chunk using PNGEncoder .

    Thank You
    Mihir Parekh
  • 3. Re: OutofMemory while loading/writing huge image
    gimbal2 Guru
    Currently Being Moderated
    987279 wrote:
    as per your second suggestion how can i cut RenderedImage into small chunks ?
    I don't know, I don't work for your company. I don't even know where the source image is coming from. That is something you'll have to figure out.
  • 4. Re: OutofMemory while loading/writing huge image
    jschellSomeoneStoleMyAlias Expert
    Currently Being Moderated
    987279 wrote:
    so for that i need (18000 * 18000 * 4 )/ (1024*1024) = 1030 MB of memory approximately
    That same equation gives me 1235 MB.
    , so probably it throws and OutofMemoryException normally. currently i am using 32 bit machine and i have 4 MB of RAM. so if i run this code on 64 bit machine with Xmx 2048 mb would i succeed to load the image without out of memory error ?
    Or a 32 bit OS (not windows) that allows access to more of the 4 gig.
    Or an older VM and and older Windows since those leave more memory for the application.

    It would load the image presuming that your values are correct.

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points