A few techniques Mustang provides:
1. Use jconsole to watch the memory usage of your application
The memory usage graph would clearly show if the memory usage is growing over time. It also provides the low memory detection support such that you can set a threshold to catch abormal behavior. For example, I set the memory usage threshold to 6 MBytes in the Tenured Gen memory pool. The amount of memory exceeding the threshold will be flagged in red color.
2. Use jconsole to monitor the garbage collection activities
The bottom panel of the memory tab displays the garbage collection statistics including the time it spent on GC and also the total number of collections happened. It would give you a rough idea of how frequent GC happens and how long the GC takes. If you want to turn on the verbose GC tracing to get details about individual GC invocation, you can do it dynamically through JConsole as described in the "Using JConsole to monitor applications" article. You don't need to restart your application.
3. Use jconsole to find out the number of objects pending for finalization
Finalizers would be one possible cause of memory leak. There is no guarantee when a finalizer will be run and whether it will be run at all. An object that has a finalizer will not be garbage collected until its finalizer is run. JConsole shows the number of objects pending for finalization in the summary tab which would be one useful information you may want to check when you suspect a leak.
4. Obtain heap histogram to help determine the leak suspect
jmap -histo will print the histogram of the heap showing the number of instances and the amount of memory of each class. jmap -histo option was extended to support on Windows in Mustang b67. The heap histogram would give you more information to determine if a memory leak would possibly exist. Often the class with growing number of instances could be one potential leak suspect.
5. Obtain heap dump snapshots and use jhat for heap analysis
Mustang enables you to get heap dump programmatically, at runtime using jmap and jconsole, at OutOfMemoryError and also from a core file. Mustang now also includes HAT (Heap Analysis Tool) as jhat utility. Check out several blogs about Mustang heap dump and jhat:
6. Prepare for OutOfMemoryError
Alan's blog describes the improvement in OutOfMemoryError and also a new VM option -XX:+HeapDumpOnOutOfMemoryError to tell the HotSpot VM to generate a heap dump when it runs out of memory. You can either specify the VM option at startup time or at runtime via jconsole's MBeans tab.
HotSpotDiagnostic MBean provides several diagnostic operations such as to dump heap and set VM option. We plan to provide a better UI for accessing the HotSpot diagnostic features. In summary, the diagnosability of memory problem has been improved. Download Mustang beta or the latest Mustang snapshotand let you know your feedback.