1 2 Previous Next


20 posts

I heard from a friend that prolific blogger and friend Felipe Gaucho had passed away last March, 05.


Click to see a larger version

Felipe was very active at Java community, helping people at mailing lists, writing blogs. He was a JUG Leader (Ceara at Brazil), Glassfish active user and speaker.

See more information at the CEJUG blog

Sorry to use this space, but I feel motivated to let the java.net community know this tragic fact that is related to an active user. Probably there are a lot of users out there that have read Felipe's  writings and they deserve to know about this terrible news.


I am working for a customer, doing some performance review for their java application and tuning glassfish as well.


They use linux 64 bits (kernel 2.6.18 SMP), glassfish v2 u2 and JDK 5 u12. At some point I needed to generate a heap dump to better understand the object allocation and if possible some hints as to where to look for optimization for the application.


Now the big problem, I could not generate a heap dump, not with the current tools available at that time. I have tried

  • -XX:+HeapDumpOnCtrlBreak and kill -3
  • jmap -heap:format=b
  • gcore utility

All of them generated an sun.jvm.hotspot.debugger.UnmappedAddressException, except gcore who just killed the process. See the stacktrace

# /usr/local/jdk/jdk1.6.0_07/bin/jmap -J-d64 -F -dump:file=java_dump_10791.hprof  10791
Attaching to process ID 10791, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 10.0-b23
Dumping heap to java_dump_10791.hprof ...
Exception in thread "main" java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at sun.tools.jmap.JMap.runTool(JMap.java:178)
        at sun.tools.jmap.JMap.main(JMap.java:110)
Caused by: sun.jvm.hotspot.debugger.UnmappedAddressException
        at sun.jvm.hotspot.debugger.PageCache.checkPage(PageCache.java:208)
        at sun.jvm.hotspot.debugger.PageCache.getData(PageCache.java:63)
        at sun.jvm.hotspot.debugger.DebuggerBase.readBytes(DebuggerBase.java:205)
        at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.readCInteger(LinuxDebuggerLocal.java:471)
        at sun.jvm.hotspot.debugger.DebuggerBase.readAddressValue(DebuggerBase.java:442)
        at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.readOopHandle(LinuxDebuggerLocal.java:431)
        at sun.jvm.hotspot.debugger.linux.LinuxAddress.getOopHandleAt(LinuxAddress.java:115)
        at sun.jvm.hotspot.oops.Oop.getKlassForOopHandle(Oop.java:222)
        at sun.jvm.hotspot.oops.ObjectHeap.newOop(ObjectHeap.java:348)
        at sun.jvm.hotspot.utilities.HashtableEntry.literal(HashtableEntry.java:53)
        at sun.jvm.hotspot.memory.SymbolTable.symbolsDo(SymbolTable.java:106)
        at sun.jvm.hotspot.utilities.HeapHprofBinWriter.writeSymbols(HeapHprofBinWriter.java:830)
        at sun.jvm.hotspot.utilities.HeapHprofBinWriter.write(HeapHprofBinWriter.java:396)
        at sun.jvm.hotspot.tools.HeapDumper.run(HeapDumper.java:56)
        at sun.jvm.hotspot.tools.Tool.start(Tool.java:221)
        at sun.jvm.hotspot.tools.HeapDumper.main(HeapDumper.java:77)

: The stacktrace displayed above, shows a dump request from a JDK 6u7 to a target VM 6 u7.
UPDATE: Previously, I had issued jmap from JDK 5 u12 to a target VM 5 u12. The stacktrace is the same.
UPDATE: The target VM is not using -Xrs option.
UPDATE: The VM process owner is the same who issued the jmap command, root.


UPDATE: Alan Bateman clarified a bit about the -F option, "The -F options causes the tool to attach to the target VM in a non-cooperative way and so may observe the heap in an inconsistent state. In other words, there is no guarantee that you will get a good heap dump when you using the -F option" read more on the comment section of my portuguese blog (the blog post is the same as posted here).


I tried JDK 6 u7, with the same error.


Later on, I found a bug "Throws UnmappedAddressException while reading address from core file in shared area."


It is fixed for JDK 6 u10 RC (changelog), so I really shoud give it a try. Don't forget the -Xshare:off option.


And it worked as it should be, after heap dump was generated the JVM works as normal.


So if somebody wants to generate heap dump on linux 64 bits, try to use JDK 6 u10 RC.


Looks like I need to say "Thank You" to Tony Printezis, as at the end of heap dump generation, there is a hint about the author's code.


"Finding object size using Printezis bits and skipping over..."



HotSpot Internals Blog

Posted by claudio Aug 20, 2008

There is a wiki site at Sun, about HotSpot Internals, with valuable information to people who wants to better understand HotSpot but don't have the proper time to dedicate to such good reading of HotSpot source code.


As an example, there are tips about how to optimize Java code to the best and how HostSpot sees it


Jetty and Solaris 10 Blog

Posted by claudio Jul 15, 2008

This is a contribution to Jetty Web Server, to allow SMF on Solaris 10 machines to manage Jetty servers, see the benefits

  • Allow non-root users to bind to privileged port
  • Watchdog service (can restart server if process is down for unknown reason)
  • Standard use of SMF service

So what is SMF ? It stands for Service Management Facility. See more information at OpenSolaris website. And a small excerpt from their website


"Self-healing services are delivered and managed on Solaris with the Service Management Facility (smf(5)). smf(5) augments the existing init.d(4) and inetd(1M) startup mechanisms, promoting the service to a first-class operating system object."


I want to let you know there is other contribution from Trygve Laugstol, where he made available a Jetty package on blastwave repository (it is outdated).


If you want to use SMF to manage Jetty, get the software (it points to the Jetty patch 639)




There is a bug on Serviceability on JDK, that can lead to problems running jmap on java process started by non-root users that can bind to privileged ports  (in fact if the process have called setuid on solaris).


This bug happens, if jmap -heap PID is called, like this example

# /opt/jdk1.6.0_07/bin/jmap -heap 21862
Attaching to process ID 21862, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 10.0-b23
concurrent mark-sweep generation:
   capacity = 125829120 (120.0MB)
Free chunk in CMS heap, size=65536
Free chunk in CMS heap, size=1048
Free chunk in CMS heap, size=1048
Free chunk in CMS heap, size=256
Free chunk in CMS heap, size=256

a lot of free chunk messages, until ctrl+c

Exception in thread "main" java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at sun.tools.jmap.JMap.runTool(JMap.java:178)
        at sun.tools.jmap.JMap.main(JMap.java:110)
Caused by: java.lang.RuntimeException: VM.initialize() was not yet called
        at sun.jvm.hotspot.runtime.VM.getVM(VM.java:359)
        at sun.jvm.hotspot.oops.ObjectHeap.newOop(ObjectHeap.java:342)
        at sun.jvm.hotspot.memory.CompactibleFreeListSpace.getLiveRegions(CompactibleFreeListSpace.java:138)
        at sun.jvm.hotspot.memory.CompactibleFreeListSpace.used(CompactibleFreeListSpace.java:68)
        at sun.jvm.hotspot.memory.ConcurrentMarkSweepGeneration.used(ConcurrentMarkSweepGeneration.java:61)
        at sun.jvm.hotspot.tools.HeapSummary.printGen(HeapSummary.java:182)
        at sun.jvm.hotspot.tools.HeapSummary.run(HeapSummary.java:94)
        at sun.jvm.hotspot.tools.Tool.start(Tool.java:221)
        at sun.jvm.hotspot.tools.HeapSummary.main(HeapSummary.java:39)


Please vote for this bug, if you want to use jmap in such conditions.

You can see my presentation about Tools and Tips to Diagnose Performance Issues on-line or download it.

It was great, because the room was almost full, people keep taking pictures, looks like the topic was interesting.

Some people asked questions at the end, where I want to answer here, to a broad audience.


All of them are hard questions. As there is no recipe to answer them.

Q.1) What is the best value for stack size (-Xss) parameter ?

Q.2) What is the best value for young generation (-XX:NewSize -XX:MaxNewSize) ?

Q.3) How to diagnose OOME issues with JDK 1.3 on windows ?



R.1) There is no "best value" it depends of the java runtime system. If possible, do some experiments, to see how it behaves for different sizes. But the lower the size, more memory efficient.


I had worked with 256k and 512k, for application servers.


Depending of the operating system and JDK, the java stack size, is the same as the operating system, see it:

$ ulimit -a|grep stack
stack size              (kbytes, -s) 8192

R.2) If possible, don't configure it, let JRE set its default value. But, as someone have asked, see it below.


If young generation needs to be customized, it means the default don't apply here.


Java runtime needs to be monitored to understand the allocation activity going on and out of young generation. Try to adjust it in a way that don't span a full garbage collector (on old generation) very often. If there is memory leak going on, a full gc can be triggered.


R.3) That is the hard one, as JDk 1.3 has reached End of Line support at Sun and lack most serviceability found in recent JDK, like jmap and jstat. With JDK 1.3 one can use some unix tools to introspect the process, but the atendee asked about windows, that is even bad, because I do not do much work on windows platform and have no knowledge about windows diagnostic tools.


What I really recommend, is to reproduce the java system in a isolated machine, use at least JDK 1.4 latest release and configure -XX:+HeapDumpOnOutOfMemoryError.


If the system is flexibe, put the application on more recent releases of JDK and servers.

Jazoon 2008

Next week I am heading up to the northern hemisphere to join a crowd of Java professionals and enthusiasts to participate to Jazoon conference.

Tools and Tips to Diagnose Performance Issues is the session I am going to share tips and tools to help diagnose performance issues on Java applications. Some of them are heap and thread dump analysys, OutOfMemoryError, system performance, monitoring and profiling. If you plan to attend Jazoon, please take a look at the session. It is going to be challenge to fit a lot of information on 50 minutes (including questions). I plan to attend other similar sessions to learn a bit more about this matter.

Do you remember the days you asked "How can I detect which class is consuming a lot of memory at the production site ?", "Which monitor is locking all threads ?", "Why are there so many http connections being queued ?", those are some answers I plan to talk about at the session.

The first day, is the community day, as there are going to happend GlassFish and NetBeans day at Monday from 9am to 3pm. The good news, it is free, but you need to register in advance.

Besides the beautiful city (which I want to spend some time outside the conference), what I find very interesting is the talks with project leaders, community members and developers from different cultures. 


If I ask someone else what a Jazooner is, probably they could think it is a character from Klingon, right ?


But for the people at Zurich organizing such a java international conference, it is a place for learning, sharing, talking about Java and meet good professionals. I am proud to be part of the show.


My session Tools and Tips to Diagnose Performance Issues, is scheduled at June, 24 at 5:30pm. At this talk I want to share with the audience, tips to help diagnose performance issues (cpu, memory, threads) and the recommended tools to use.


See the complete presentation program and speakers's list.    


You can see detailed information at its website andChristian Frei's blog (co-organizer of Jazoon).    


3 days conference pass for free


Jazoon is offering this free pass, to selected speaker who submit sessions entiled for the category "Jazoon Cutting-Edge", 15 min talks about thelatest trends, updates and highlights of the Java world.    



The call for papers open from May, 01 to May, 16. See the website conference for more information.      


A long time ago I have developed a plugin to NetBeans to graphically configure netbeans.conf file. The plugin is "NetBeans Startup Settings".


Use the comments' section to give feedback. Or send me an email claudio @@ dev.java.net 


NetBeans plugin portal page


project page at javaforge.com


NetBeans Plugin information




Currently to add or change any NetBeans parameter (font size, jvm argument, look and feel), a etc/netbeans.conf file needs to be edited.


So this module, can make it easy to change those parameters, see below screenshot and feature list:


Feature list

  • Backup and restore original settings (etc/netbeans.conf)
  • Able to configure and store different JDK and NetBeans user directory
  • Store settings (jdk and userdir) under JDK Preference API
  • Support to on-line updates through update center
  • LGPL license
  • If specified userdir doesn't exist, a new one will be created


  • JDK 5 or higher
  • NetBeans 6.x
  • A writable $HOME/.java/.userPrefs directory
Currently, it is beta quality: 0.1 beta  

download     the NBM        

see the changelog  


Last year when working in a project, there were a lot of documents (requirements, user guides, architecture, etc.), from different sources (email attachments, file shares, backups, old version control). The same document name but different date and size.

So, how to know which one is the latest and delete the others ?

There were two ways to achieve that:

  1. Open each file and see its properties, like date and last author.
  3. Programmatically read its properties and print it out.
And there are their costs:
  1. Very time consuming if you have many files, like I had (300 files)
  3. Time consuming, but you do this only once and can write a blog about that (WoW)
Sure, I went to the 2nd option.

Then, in a short time I developed a small program to read a set of files and print its name, date of last modification and complete path.

OpenOffice SDK libraries is used, so you need to have a OpenOffice 2.x installed somewhere. Actually it reads a small set of office files like sxw, doc, xls, odt, ods, pps, odt, ppt and odp. Feel free to modify it to suit your needs. Or even extend its functionality.

The java source code can be downloaded at DocViewer.java  
* its a UTF-8 file  
** remove .txt extension after download

As I am a linux user, this works with linux in mind. Windows users have reported it works with small modifications to its runtime settings, but I don't know which modifications need to be done. 


Compile time

The following libraries are needed:


$OO_HOME points to the OpenOffice installation. For me it is installed at /opt/broffice.org2.3

* BrOffice is the official Brazilian version of OpenOffice

At runtime


  • OpenOffice 2.x installation  
  • X Virtual Frame Buffer (Xvfb)
  • Java (version 5 or more recent)


Very easy to compile

javac -classpath /opt/broffice.org2.3/program/classes/\* src/claudius/DocViewer.java

You see I have used classpath wildcards. Modify this as needed to compile it with JDK 5.  


To run it, openoffice standalone program need to be running, but to avoid a graphical program popping out in a window hundreds of times, it can run in a non graphical way. To achieve that I used X Virtual Frame Buffer (xvfb), its a kind of X window manager in memory, this is useful to run graphical libraries at server machines.

The OpenOffice SDK will connect to OpenOffice program through sockets, as the standalone program will do the real job of read the office file. 

  1. Run the Xvfb program  
    Xvfb :5 -screen 0 800x600x16 &

    Load openoffice program on memory and inform it to use the X server at :5 display    

    $OO_HOME/program/soffice -accept="socket,host=,port=8100;urp;" -display :5 -headless -norestore -invisible &
  5. Run the java program  
    java -classpath $CP claudius.DocViewer <path>

    $CP point to the classpath defined previously    
    <path> point to a single file or directory. If a directory it will search at subdirectories.  


The output will look similar to this


dir  = /home/claudio/resources/palestras/2007/10_justjava
file = diagnostico2.odp
Modified by: Claudio Miranda 5/10/2007 17:46:8

If this piece of code is useful or if you made any modification, please share it and write a comment.

A customer asked me to verify how many threads is possible to run concurrently at any java vm on the server. I did some search at the customer support site, but was not lucky to found any proper test utility, so I created one and hope it be useful to someone else.

At first I thought it was just a matter of a for loop, create and start the thread inside the loop. Easy task.

As I ran the test with 1k, 10k threads I saw that many threads finished before others were even created, so I needed to first create all thread object, without starting it. Wow, that is not easy to create such a mechanism, but for my lucky there is the concurrent library from Doug Lea, from there I use the CyclicBarrier class.

As that was not enough, the processing task needed to be intensive, then some Random class were created and math operations at them.

The utility class can be run from command line and deployed as war application. It is compatible to run under JDK 1.4 and Java EE 1.3.

As JDK 1.4 is a required platform, I could not use Java Concurrent API, but used the util.concurrent from Doug Lea.

Take a look at the readme.txt file, there are some useful information there.

Download and test the thread capacity application.  
Update (2007/05/24): The source code  


This utility test the ability to create many threads as possible per java process, 
as the user request it by command line or http parameter.

This test can be run from command line or deployed as an war file.

The reason to test it from a servlet container is to see the thread limit per 
appserver instance, as each instance have different JVM tuning from command line.

Beware that the maximum number of threads can be different when running from 
command line and servlet container. 
* There are some JVM parameters that cannot be tuned as the command line, 
    such as -Xmx and -Xss
* At servlet container there are thread pools and thread management


JDK 1.4, 1.5 and 6
Java EE 1.3 and 1.4


package util.concurrent (Doug Lea)


english (en)
portuguese (pt_BR)

By default the application will use the system default. If you want to specify different locale
use the environment variable (unix) LANG before invoking the script, example:

LANG=en ./start_thread_test.sh

At the AppServer, use the java properties -Duser.language and -Duser.country


First, all thread objects is created (new Thread)

A CyclicBarrier is used to control the exact moment, when all thread objects are 
ready to run.

All threads are started, but its run method has a barrier to wait until all 
threads have its method start() invoked.

The reasoning is to have all threads to run concurrently.

After all threads are properly running, there is a join() call on each thread, to
make the main method wait, until all threads run sucessfully or are interrupted 
by some error.


As this is a test of resource usage, be careful because the system can be
unresponsive at all.

The war file has authentication disabled, to enable it, edit web.xml and uncomment
the security section.

Its possible to run this test on public accessible machine, for this to work, 
create any user at the file-realm and associate it with the group named "threadtest"


To use this thread test, you can invoke the start_thread_test.sh script.

First, you need to unpack the war file. I recommend to create a new directory 
and unpack it there.

mkdir thread_test
cd thread_test
jar xf ../thread-capacity-web.war
chmod +x *.sh

./start_thread_test.sh [n] [thread repeat]

[n] is the desired number of threads (default = 15000)
[thread repeat] the number of interactions inside the thread (default = 100)


The number of threads can be monitored through operating system tools, such as:

* Linux: ps -p PID -o pid,user,%cpu,rss,etime,nlwp,args
    The NLWP column display the number of threads

* Solaris: prstat -c -p PID 1 100

At the root of war file there are some better shell scripts to monitor and run
this utility

./linux_threads_per_process.sh updatecenter  300


threads_per_process.sh PID | process name [count] "

  PID: 36434 OR
  process string: NumThreads (this script will do a ps -ef|grep NumThreads to get the PID)
  The last number is the number of times the command will run

     threads_per_process.sh 36434 20"
     threads_per_process.sh 36434 "
     threads_per_process.sh NumThreads"
     threads_per_process.sh NumThreads 20"

Last month a friend of mine asked some tips on what to do to migrate an application running on top of weblogic 6.x to version 9.

As I have previous experience migrating application from/to Sun AppServer and JBoss, I thought it should be useful to someone else, if you have any tip, please write it on the comments section or send me an email, so that I can update this blog.

This is just a bunch of tips, not exclusively related to weblogic, but can be applied to any appserver. And they are not sorted in any important order.

The tips are organized as this example:

  1. This is the tip  
    The explanation about this tip.  

Here we go:


  1. Have access to the running system, the system where the application is running on.  
    To see the settings of: database, queues, JVM parameters, permissions, etc.  
  3. Have full access to the new system, where it will be deployed  
    It need to be a system where the stop/start/debugging is not a problem.  
  5. Have a way to use the application and test it on the new system.  
    It can be done by someone else who know the application.  
  7. Have full access to the source code and build files  
    Some circunstances can lead to change the code: use of vendor APIs that is deprecated; hard coded paths; threads, permission and classloaders issues.  
  9. See at the vendor website, documentation or tools to help the migration task.  
  11. Create the deployment descriptors of the new appserver version.  
  13. Configure the connection pools, queues, factories, JVM parameters, policy permissions, SSL, security on the new system  
  15. Configure the log level to the finest when some problems occurs.  
  17. Check if all the application's libraries is compatible to the appserver and JVM version.  
    Some classloaders and policy permission can happen  
  19. Document every configuration, changes, etc.  
    This is very important, as it can be used by others or to configure the production servers.  

As an example, I have migrated (in 2005) a huge application that used:  

  • EJB (Stateless and Stateful) 1.1
  • IBM MQ Queues,
  • JAAS LDAP Authentication
  • Servlet timers
  • Distributed Transactions (Oracle and IBM MQ)
  • SSL Certificates (client and server)
  • EJBs deployed as modules and inside .ears

NetBeans Dream Team ? Blog

Posted by claudio Nov 4, 2006
Sun and NetBeans are putting a tremendous effort to put NetBeans IDE/Platform on the lead of the IDE race. I don't known exactly when this effort started, but I suppose it was at 4.1, since then a lot of energy/blogging/tutorials/releases came out to demonstrate the power of NetBeans. As I am subscribed to netbeans mailing lists, I see how the number of participants  got larger at every release, much more questions and bug reporting. Afff, the community is getting bigger. So big that they are choosing which community member will be part of NetBeans Dream Team. http://www.netbeans.org/community/contribute/dreamteam.html If you think you apply for the slot, be fast, the doors are open until nov, 17. I see the link at the community page, but didn't see any blog entry related to that such important event, so be here.

Java LiveCD Blog

Posted by claudio Jun 25, 2006

NetBeans 5.5 and JavaDB Blog

Posted by claudio May 11, 2006
NetBeans 5.5 (beta) doesn't make available by default the JavaDB (based on Apache Derby) on to Tools menu. Note, the Enterprise Pack is not installed. If you have Glassfish or Sun Java System App Server 9, then JavaDB is already there. Look at $GLASSFISH_HOME/javadb. If not, download it from JavaDB or Apache Derby. The JavaDB is available at Development Update Center or Enterprise Pack Update Center. As I am using the 5.5 beta, then the JavaDB is not available, but the module to start/stop is there. Go to NetBeans Advanced Options: Tools menu --> Options --> Advanced Options (bottom left button) --> Look at the below screenshot: Configure the JavaDB installation directory and the directory used to store the database nb55-javadb-start.png Now you can start/stop JavaDB from Tools menu nb55-javadb-start.png

Filter Blog

By date: