This discussion is archived
5 Replies Latest reply: Nov 4, 2009 7:54 PM by 843829 RSS

URGENT: upgrade from jdk 1.5 to 1.6 causes OutOfMemoryErrors

843829 Newbie
Currently Being Moderated
I have an application that inserts several million records into a database.

Everything works find under JDK 1.4 and 1.5 and according to JProfiler, no memory leaks exist. the heap usage never exceeds 20MB when running under either of these jvms.

but for some reason, memory consumption under Java 1.6 skyrockets. the problem occurs with various Java 1.6.0_XX versions on Windows as well as on Linux. it also occurs regardless of database (oracle 10g or mysql 5.0.45) which pretty much rules out the jdbc driver causing the problem.

the jvm is launched without any extra parameters (aside from -XX:+PrintGCDetails).

The application uses Spring 1.2 and Hibernate 3.2 (although neither of these appears to be responsible for the massive memory consumption).

i have experimented with both the serial and parallel garbage collectors, tried increasing eden (young) size, tried increasing the jvm heap size (-Xmx; this simply delays the OutOfMemoryError) but nothing seems to solve the problem. i have included snippets of the GC details when running under either JVM.

im pretty much out of theories as to how to solve this. does anyone have any clue?

much appreciated
Benjamin

Here is a sample of the GC details when running under Java 1.5
------------------------------------------------------------------------------------------------------------------------
[GC [DefNew: 704K->63K(704K), 0.0009842 secs][Tenured: 8741K->4819K(8852K), 0.0673614 secs] 9254K->4819K(9556K), 0.0684909 secs]
[GC [DefNew: 640K->64K(704K), 0.0010998 secs] 5459K->4961K(9428K), 0.0013923 secs]
[GC [DefNew: 704K->63K(704K), 0.0008307 secs] 5601K->5128K(9428K), 0.0009015 secs]
[GC [DefNew: 703K->63K(704K), 0.0009454 secs] 5768K->5314K(9428K), 0.0010974 secs]
[GC [DefNew: 703K->64K(704K), 0.0005796 secs] 5954K->5365K(9428K), 0.0006691 secs]
[GC [DefNew: 704K->64K(704K), 0.0005730 secs] 6005K->5452K(9428K), 0.0006394 secs]
[GC [DefNew: 704K->64K(704K), 0.0008441 secs] 6092K->5630K(9428K), 0.0009199 secs]
[GC [DefNew: 704K->63K(704K), 0.0009119 secs] 6270K->5676K(9428K), 0.0011003 secs]
[GC [DefNew: 703K->64K(704K), 0.0008150 secs] 6316K->5761K(9428K), 0.0010778 secs]
[GC [DefNew: 704K->63K(704K), 0.0012583 secs] 6401K->5930K(9428K), 0.0015591 secs]
[GC [DefNew: 703K->63K(704K), 0.0005651 secs] 6570K->5983K(9428K), 0.0006504 secs]
[GC [DefNew: 703K->63K(704K), 0.0006776 secs] 6623K->6063K(9428K), 0.0007520 secs]
[GC [DefNew: 703K->64K(704K), 0.0010686 secs] 6703K->6225K(9428K), 0.0011714 secs]
[GC [DefNew: 704K->63K(704K), 0.0012748 secs] 6865K->6277K(9428K), 0.0015974 secs]
[GC [DefNew: 703K->64K(704K), 0.0006470 secs] 6917K->6355K(9428K), 0.0007239 secs]
[GC [DefNew: 704K->63K(704K), 0.0008335 secs] 6995K->6510K(9428K), 0.0008702 secs]
[GC [DefNew: 703K->63K(704K), 0.0005282 secs] 7150K->6560K(9428K), 0.0005635 secs]
[GC [DefNew: 703K->64K(704K), 0.0005776 secs] 7200K->6634K(9428K), 0.0006132 secs]
[GC [DefNew: 704K->64K(704K), 0.0009024 secs] 7274K->6786K(9428K), 0.0009403 secs]
[GC [DefNew: 704K->64K(704K), 0.0009929 secs] 7426K->6916K(9428K), 0.0010620 secs]
[GC [DefNew: 704K->64K(704K), 0.0007581 secs] 7556K->6993K(9428K), 0.0009915 secs]
[GC [DefNew: 704K->63K(704K), 0.0008686 secs] 7633K->7147K(9428K), 0.0009058 secs]
[GC [DefNew: 703K->64K(704K), 0.0008011 secs] 7787K->7277K(9428K), 0.0008358 secs]
[GC [DefNew: 704K->63K(704K), 0.0005985 secs] 7917K->7351K(9428K), 0.0006325 secs]
[GC [DefNew: 703K->64K(704K), 0.0011532 secs] 7991K->7504K(9428K), 0.0012416 secs]
[GC [DefNew: 704K->63K(704K), 0.0010324 secs] 8144K->7636K(9428K), 0.0011210 secs]
[GC [DefNew: 703K->64K(704K), 0.0006600 secs] 8276K->7707K(9428K), 0.0007347 secs]
[GC [DefNew: 704K->63K(704K), 0.0008733 secs] 8347K->7859K(9428K), 0.0009091 secs]
[GC [DefNew: 683K->64K(704K), 0.0011623 secs] 8479K->8066K(9428K), 0.0012492 secs]
[GC [DefNew: 704K->63K(704K), 0.0006170 secs] 8706K->8129K(9428K), 0.0006887 secs]
[GC [DefNew: 703K->63K(704K), 0.0008206 secs] 8769K->8234K(9428K), 0.0009041 secs]
[GC [DefNew: 703K->63K(704K), 0.0008461 secs] 8874K->8434K(9428K), 0.0008814 secs]
[GC [DefNew: 703K->63K(704K), 0.0005558 secs] 9074K->8489K(9428K), 0.0006446 secs]
[GC [DefNew: 703K->64K(704K), 0.0006280 secs] 9129K->8584K(9428K), 0.0007199 secs]
[GC [DefNew: 697K->64K(704K), 0.0009556 secs] 9217K->8768K(9428K), 0.0010326 secs]
[GC [DefNew: 704K->63K(704K), 0.0005956 secs][Tenured: 8758K->4703K(8852K), 0.0646623 secs] 9408K->4703K(9556K), 0.0654001 secs]



Here is a sample of the GC details when running under Java 1.6
------------------------------------------------------------------------------------------------------------------------
[GC [DefNew: 918K->64K(960K), 0.0020567 secs] 3778K->3020K(5056K), 0.0021595 secs]
[GC [DefNew: 949K->42K(960K), 0.0014172 secs] 3905K->3057K(5056K), 0.0015471 secs]
[GC [DefNew: 927K->35K(960K), 0.0010390 secs] 3942K->3086K(5056K), 0.0011432 secs]
[GC [DefNew: 833K->64K(960K), 0.0015407 secs] 3884K->3374K(5056K), 0.0016432 secs]
[GC [DefNew: 950K->23K(960K), 0.0012706 secs] 4261K->3500K(5056K), 0.0013968 secs]
[GC [DefNew: 919K->36K(960K), 0.0011697 secs] 4396K->3513K(5056K), 0.0012770 secs]
[GC [DefNew: 929K->55K(960K), 0.0012376 secs] 4407K->3551K(5056K), 0.0013605 secs]
[GC [DefNew: 951K->64K(960K), 0.0014868 secs] 4447K->3590K(5056K), 0.0015946 secs]
[GC [DefNew: 960K->64K(960K), 0.0017441 secs] 4486K->3701K(5056K), 0.0018505 secs]
[GC [DefNew: 960K->64K(960K), 0.0020746 secs] 4597K->3796K(5056K), 0.0021760 secs]
[GC [DefNew: 960K->6K(960K), 0.0015848 secs] 4692K->3757K(5056K), 0.0016879 secs]
[GC [DefNew: 898K->38K(960K), 0.0010172 secs] 4649K->3789K(5056K), 0.0011267 secs]
[GC [DefNew: 934K->57K(960K), 0.0010758 secs] 4685K->3852K(5056K), 0.0011904 secs]
[GC [DefNew: 912K->64K(960K), 0.0012502 secs] 4706K->3997K(5056K), 0.0014088 secs]
[GC [DefNew: 927K->46K(960K), 0.0011865 secs] 4860K->3998K(5056K), 0.0013099 secs]
[GC [DefNew: 942K->64K(960K), 0.0012960 secs][Tenured: 4089K->3401K(4096K), 0.0425501 secs] 4894K->3401K(5056K), 0.0441511 secs]
[GC [DefNew: 896K->33K(960K), 0.0011024 secs] 4297K->3435K(6632K), 0.0012661 secs]
[GC [DefNew: 925K->41K(960K), 0.0009093 secs] 4327K->3458K(6632K), 0.0010124 secs]
Processing file: xpressfeed_files\northamerica\f_sec_dprc.1.xml
[GC [DefNew: 937K->62K(960K), 0.0013139 secs] 4354K->3503K(6632K), 0.0014167 secs]
[GC [DefNew: 958K->63K(960K), 0.0016036 secs] 4399K->3571K(6632K), 0.0017231 secs]
[GC [DefNew: 925K->64K(960K), 0.0020497 secs] 4433K->4047K(6632K), 0.0021545 secs]
[GC [DefNew: 960K->63K(960K), 0.0028434 secs] 4943K->4682K(6632K), 0.0029479 secs]
[GC [DefNew: 949K->64K(960K), 0.0034273 secs] 5567K->5432K(6632K), 0.0035356 secs]
[GC [DefNew: 761K->63K(960K), 0.0027110 secs][Tenured: 5760K->3870K(5800K), 0.0521882 secs] 6130K->3870K(6760K), 0.0552171 secs]
[GC [DefNew: 896K->63K(960K), 0.0015790 secs] 4766K->4660K(7412K), 0.0016837 secs]
[GC [DefNew: 959K->64K(960K), 0.0022053 secs] 5556K->5198K(7412K), 0.0023537 secs]
[GC [DefNew: 960K->64K(960K), 0.0047609 secs][Tenured: 6672K->5573K(6708K), 0.0663528 secs] 7134K->5573K(7668K), 0.0714929 secs]
[GC [DefNew: 896K->64K(960K), 0.0013912 secs] 8550K->7829K(10252K), 0.0014999 secs]
[GC [DefNew: 960K->0K(960K), 0.0009803 secs] 8725K->7829K(10252K), 0.0010817 secs]
[GC [DefNew: 658K->0K(960K), 0.0003542 secs][Tenured: 7829K->6726K(9292K), 0.0533249 secs] 8488K->6726K(10252K), 0.0539479 secs]
[GC [DefNew: 896K->0K(960K), 0.0003906 secs] 11782K->10886K(12172K), 0.0005096 secs]
[GC [DefNew: 896K->0K(960K), 0.0003760 secs] 11782K->10886K(12172K), 0.0004794 secs]
[GC [DefNew: 896K->0K(960K), 0.0003961 secs] 11782K->10886K(12172K), 0.0005006 secs]
[GC [DefNew: 550K->0K(960K), 0.0003816 secs][Tenured: 10886K->8806K(11212K), 0.0578118 secs] 11437K->8806K(12172K), 0.0587164 secs]
[GC [DefNew: 1024K->0K(1152K), 0.0006045 secs] 18150K->17126K(24156K), 0.0007713 secs]
[GC [DefNew: 1600K->0K(1792K), 0.0004400 secs] 18726K->17126K(24796K), 0.0005490 secs]
[GC [DefNew: 1600K->0K(1792K), 0.0005157 secs] 18726K->17126K(24796K), 0.0006610 secs]
[GC [DefNew: 1600K->0K(1792K), 0.0003872 secs] 18726K->17126K(24796K), 0.0004886 secs]
[GC [DefNew: 635K->0K(1792K), 0.0004294 secs][Tenured: 17126K->12966K(23004K), 0.0691820 secs] 17761K->12966K(24796K), 0.0697843 secs]
[GC [DefNew: 1600K->0K(1792K), 0.0004238 secs] 31206K->29606K(41440K), 0.0005246 secs]
[GC [DefNew: 1600K->0K(1792K), 0.0004216 secs] 31206K->29606K(41440K), 0.0006283 secs]
[GC [DefNew: 2688K->0K(3008K), 0.0004735 secs] 32294K->29606K(42656K), 0.0005802 secs]
[GC [DefNew: 2688K->0K(3008K), 0.0004411 secs] 32294K->29606K(42656K), 0.0005629 secs]
[GC [DefNew: 2688K->0K(3008K), 0.0005797 secs] 32294K->29606K(42656K), 0.0006856 secs]
[GC [DefNew: 1628K->0K(3008K), 0.0004562 secs][Tenured: 29606K->21253K(39648K), 0.0685190 secs] 31235K->21253K(42656K), 0.0691471 secs]
  • 1. Re: URGENT: upgrade from jdk 1.5 to 1.6 causes OutOfMemoryErrors
    843829 Newbie
    Currently Being Moderated
    1. Can you post full error message for OOM and full stack trace?


    2. You said JProfiler showed no evidence of memory leak on Jdk 1.4 and 1.5. How about Jdk 1.6? What does JProfiler show?
  • 2. Re: URGENT: upgrade from jdk 1.5 to 1.6 causes OutOfMemoryErrors
    843829 Newbie
    Currently Being Moderated
    thanks for showing interest in this problem

    i have included the the OOM stack trace below. its the traditional Java Heap Space error...

    With JProfiler, the app never goes over 20MB of usage when running under jre 1.5. But then with 1.6, usage skyrockets with the "char[]" class being responsible for some 200 MB of memory (running with -Xms128m -Xmx256m). shortly thereafter the thread crashes.

    The application simply parses (very large) XML files using the SAX parser supplied by Sun in the JRE/JDK...in this case its Apache's Xerces implementation.

    Here are links to some images taken from JProfiler. If you think any others would be of assistance just tell me and i can post them as well.
    http://www.javenue.org/mem_profile_jre1.6.JPG
    http://www.javenue.org/object_profile_jre1.6.JPG
    http://www.javenue.org/allocated_objects_jre1.6.JPG


    OOM StackTrace:
    -----------------------------------------
    Caused by: java.lang.OutOfMemoryError: Java heap space
         at com.sun.org.apache.xerces.internal.util.XMLStringBuffer.append(Unknown Source)
         at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.refresh(Unknown Source)
         at com.sun.org.apache.xerces.internal.impl.XMLEntityScanner.invokeListeners(Unknown Source)
         at com.sun.org.apache.xerces.internal.impl.XMLEntityScanner.skipChar(Unknown Source)
         at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(Unknown Source)
         at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(Unknown Source)
         at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
         at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown Source)
         at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown Source)
         at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(Unknown Source)
         at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(Unknown Source)
         at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source)
         at javax.xml.parsers.SAXParser.parse(Unknown Source)
         at javax.xml.parsers.SAXParser.parse(Unknown Source)
         ... 6 more
  • 3. Re: URGENT: upgrade from jdk 1.5 to 1.6 causes OutOfMemoryErrors
    843829 Newbie
    Currently Being Moderated
    I think i found the problem. It appears to be a bug in the SAX implementation in the 1.6 JVM.

    here is a link to the bug description:

    http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6536111

    seems we'll just have to wait for Sun to fix this....

    i rewarded you (Dion_Cho) with 4 of the 10 duke stars since your request to have me post the stacktrace led me to try google-ing the xerces parser under java 1.6. thanks for the tip ;)

    Edited by: bennini on Nov 19, 2007 9:34 AM
  • 4. Re: URGENT: upgrade from jdk 1.5 to 1.6 causes OutOfMemoryErrors
    805006 Newbie
    Currently Being Moderated
    I get the following error message with the SAX parser. It seems to be a bug in SAX parser. I research with the JavaVisualVM and char types seems to be used too much. SAX parses large XML files. (1GB - 15GB)

    Bug report (http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6536111) says it was fixed and delivered. But it seems to be same problem. Have you got any solution? Or idea?

    Thanks

    I use java 1.6 update 11.
    OS is win XP SP 2.0
    Xerces SAX 2.9
    Exception in thread "Thread-4" java.lang.OutOfMemoryError: Java heap space
         at com.sun.org.apache.xerces.internal.util.XMLStringBuffer.append(Unknown Source)
         at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.refresh(Unknown Source)
         at com.sun.org.apache.xerces.internal.impl.XMLEntityScanner.invokeListeners(Unknown Source)
         at com.sun.org.apache.xerces.internal.impl.XMLEntityScanner.skipChar(Unknown Source)
         at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(Unknown Source)
         at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(Unknown Source)
         at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(Unknown Source)
         at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
         at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown Source)
         at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown Source)
         at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(Unknown Source)
         at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(Unknown Source)
         at abc.util.XMLParserUtil.parse(XMLParserUtil.java:39)
         at abc.clientvalidation.FFEngine.startProcess(FFEngine.java:57)
         at abc.clientvalidation.FFMainFrame$actListener$1.run(FFMainFrame.java:235)
  • 5. Re: URGENT: upgrade from jdk 1.5 to 1.6 causes OutOfMemoryErrors
    843829 Newbie
    Currently Being Moderated
    Even I am facing the same error in jdk6_16 release. Is this issue really fixed ??