Hi all,
I created a flash based walk-through.
Check my blog or go directly to http://img301.imageshack.us/my.php?image=mem1vj6.swf&width=1024

Regards,
Markus

Hi,
SDN itself runs on IBM 1.4 JDK with NW04 SPS18 on Linux64 Bit.
Do you also support IBM JDK or have you just not tested it?
Is it worth a try?
Thanks,
M.

Hi Markus,

Sorry, but IBM has its own heap dump format or more preciely, its own API (DTFJ) for dumps as they don't publish a format spec, but an API. You are bound to IBM tools for analysis, only.

However, most likely it is possible to write a converter to HPROF binary heap dumps - the API seems to offer all necessary information. But, I don't know of anybody who has written such a converter, be it commercial or open source.

Kind regards,

Vedran

Hi,

 Has anyone come across a NPE when trying to load a heap dump?

 I get the following :

!SESSION 2007-07-02 13:45:02.848 -----------------------------------------------
eclipse.buildId=I20070608-1718
java.version=1.6.0_01
java.vendor=Sun Microsystems Inc.
BootLoader constants: OS=linux, ARCH=x86_64, WS=gtk, NL=en_GB
Command-line arguments:  -os linux -ws gtk -arch x86_64

!ENTRY com.sap.tools.memory.ui.core 1 0 2007-07-02 13:47:37.247
!MESSAGE Heap /sbcimp/dyn/logfiles/jetpac/MS-TFI1-JeT_Stm2.hprof contains 36,440,149 objects

!ENTRY org.eclipse.core.jobs 4 2 2007-07-02 13:57:59.952
!MESSAGE An internal error occurred during: "parsing /sbcimp/dyn/logfiles/jetpac/MS-TFI1-JeT_Stm2.hprof".
!STACK 0
java.lang.NullPointerException
        at com.sap.tools.memory.snapshot.hprof.HprofParserHandlerImpl.cleanupGarbage(HprofParserHandlerImpl.java:501)
        at com.sap.tools.memory.snapshot.hprof.HprofParserHandlerImpl.createSnapshot(HprofParserHandlerImpl.java:321)
        at com.sap.tools.memory.snapshot.SnapshotFactory.parse(SnapshotFactory.java:402)
        at com.sap.tools.memory.snapshot.SnapshotFactory.createSnapshot(SnapshotFactory.java:113)
        at com.sap.tools.memory.ui.core.internal.ParseHeapDumpJob.run(ParseHeapDumpJob.java:40)
        at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55)

Hi Scott,

let's have a closer look. Unfortunately, you haven't exposed any contact information, e.g. an e-mail address in your SDN account. We can't contact you for the heap dump or any further information, but we certainly would like to get hold of your problem and solve it.

Can you please e-mail either to me (vedran.lerenc@sap.com) or anybody else on the Wiki page listed as contacts?

And for all others with problems: Feel free to e-mail us. We will see what we can do for you.

Thank you in advance,

Vedran

Hi Scott,

I have analyzed the heap dump you sent me, and found that the problems have appeared, because the format of the hprof dump generated by using jmap had some differences (in my opinion incorrect) compared to the dumps the VMs on the different platforms produce on OutOfMemoryError or on ctrl_break event.

I have implemented some changes, so that our future versions are able to work with jmap-produced dumps despite of the mentioned differences.

Thank you for pointing us to the problem.

Regards, Krum 

Hi,

First thanks for providing such a great tool. I was able to load a 3.6GB heap dump (generated on Solaris) onto my 32-bit XP machine and investigate a number of problems.

A couple of errors were reported by the tool while running it. Reporting them all here in the hopes they can be addressed in the next public version.

1. Error: Unhandled event loop exception. An exception stack trace is not available.

2. java.lang.NullPointerException
at com.sap.tools.memory.ui.views.tree.ObjectLeafNode$4.compare(ObjectLeafNode.java:162)
at com.sap.tools.memory.ui.views.tree.ObjectLeafNode$4.compare(ObjectLeafNode.java:160)
at java.util.Arrays.mergeSort(Arrays.java:1284)
at java.util.Arrays.sort(Arrays.java:1265)
at com.sap.tools.memory.ui.core.tree.TreeNodeArray.sort(TreeNodeArray.java:75)
at com.sap.tools.memory.ui.views.editors.ObjectsListPane$ObjectNode.sortChildren(ObjectsListPane.java:253)
at com.sap.tools.memory.ui.views.editors.ObjectsListPane$ObjectNode.load(ObjectsListPane.java:137)
at com.sap.tools.memory.ui.core.tree.TreeContentProvider.getChildren(TreeContentProvider.java:154)
at org.eclipse.jface.viewers.AbstractTreeViewer.getRawChildren(AbstractTreeViewer.java:1157)
at org.eclipse.jface.viewers.TreeViewer.getRawChildren(TreeViewer.java:768)
at org.eclipse.jface.viewers.AbstractTreeViewer.getFilteredChildren(AbstractTreeViewer.java:565)
at org.eclipse.jface.viewers.AbstractTreeViewer.getSortedChildren(AbstractTreeViewer.java:534)
at org.eclipse.jface.viewers.AbstractTreeViewer$1.run(AbstractTreeViewer.java:719)
at org.eclipse.swt.custom.BusyIndicator.showWhile(BusyIndicator.java:67)
at org.eclipse.jface.viewers.AbstractTreeViewer.createChildren(AbstractTreeViewer.java:696)
at org.eclipse.jface.viewers.TreeViewer.createChildren(TreeViewer.java:892)
at org.eclipse.jface.viewers.AbstractTreeViewer.handleTreeExpand(AbstractTreeViewer.java:1240)
at org.eclipse.jface.viewers.AbstractTreeViewer$4.treeExpanded(AbstractTreeViewer.java:1252)
at org.eclipse.swt.widgets.TypedListener.handleEvent(TypedListener.java:181)
at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:66)
at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:928)
at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:952)
at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:937)
at org.eclipse.swt.widgets.Tree.wmNotifyChild(Tree.java:6254)
at org.eclipse.swt.widgets.Control.WM_NOTIFY(Control.java:3794)
at org.eclipse.swt.widgets.Composite.WM_NOTIFY(Composite.java:1166)
at org.eclipse.swt.widgets.Control.windowProc(Control.java:3298)
at org.eclipse.swt.widgets.Display.windowProc(Display.java:4025)
at org.eclipse.swt.internal.win32.OS.CallWindowProcW(Native Method)
at org.eclipse.swt.internal.win32.OS.CallWindowProc(OS.java:1842)
at org.eclipse.swt.widgets.Tree.callWindowProc(Tree.java:1319)
at org.eclipse.swt.widgets.Tree.WM_LBUTTONDOWN(Tree.java:5171)
at org.eclipse.swt.widgets.Control.windowProc(Control.java:3279)
at org.eclipse.swt.widgets.Tree.windowProc(Tree.java:4768)
at org.eclipse.swt.widgets.Display.windowProc(Display.java:4025)
at org.eclipse.swt.internal.win32.OS.DispatchMessageW(Native Method)
at org.eclipse.swt.internal.win32.OS.DispatchMessage(OS.java:1923)
at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:2966)
at org.eclipse.ui.internal.Workbench.runEventLoop(Workbench.java:1914)
at org.eclipse.ui.internal.Workbench.runUI(Workbench.java:1878)
at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:419)
at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:149)
at com.sap.tools.memory.ui.rcp.Application.run(Application.java:14)
at org.eclipse.core.internal.runtime.PlatformActivator$1.run(PlatformActivator.java:78)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:92)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:68)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:400)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:177)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.eclipse.core.launcher.Main.invokeFramework(Main.java:336)
at org.eclipse.core.launcher.Main.basicRun(Main.java:280)
at org.eclipse.core.launcher.Main.run(Main.java:977)
at org.eclipse.core.launcher.Main.main(Main.java:952)

3. Problems occurred when invoking code from plug-in: "org.eclipse.jface".
java.lang.NullPointerException
at com.sap.tools.memory.ui.views.tree.ObjectLeafNode$4.compare(ObjectLeafNode.java:162)
at com.sap.tools.memory.ui.views.tree.ObjectLeafNode$4.compare(ObjectLeafNode.java:160)
at java.util.Arrays.mergeSort(Arrays.java:1284)
at java.util.Arrays.mergeSort(Arrays.java:1295)
at java.util.Arrays.mergeSort(Arrays.java:1295)
at java.util.Arrays.mergeSort(Arrays.java:1295)
at java.util.Arrays.mergeSort(Arrays.java:1295)
at java.util.Arrays.mergeSort(Arrays.java:1295)
at java.util.Arrays.mergeSort(Arrays.java:1295)
at java.util.Arrays.mergeSort(Arrays.java:1295)
at java.util.Arrays.mergeSort(Arrays.java:1295)
at java.util.Arrays.mergeSort(Arrays.java:1295)
at java.util.Arrays.mergeSort(Arrays.java:1295)
at java.util.Arrays.mergeSort(Arrays.java:1295)
at java.util.Arrays.mergeSort(Arrays.java:1295)
at java.util.Arrays.mergeSort(Arrays.java:1295)
at java.util.Arrays.sort(Arrays.java:1265)
at com.sap.tools.memory.ui.core.tree.TreeNodeArray.sort(TreeNodeArray.java:75)
at com.sap.tools.memory.ui.views.editors.ObjectsListPane$ObjectNode.sortChildren(ObjectsListPane.java:253)
at com.sap.tools.memory.ui.views.editors.ObjectsListPane$ObjectNode.load(ObjectsListPane.java:137)
at com.sap.tools.memory.ui.core.tree.TreeContentProvider.getChildren(TreeContentProvider.java:154)
at org.eclipse.jface.viewers.AbstractTreeViewer.getRawChildren(AbstractTreeViewer.java:1157)
at org.eclipse.jface.viewers.TreeViewer.getRawChildren(TreeViewer.java:768)
at org.eclipse.jface.viewers.AbstractTreeViewer.getFilteredChildren(AbstractTreeViewer.java:565)
at org.eclipse.jface.viewers.AbstractTreeViewer.getSortedChildren(AbstractTreeViewer.java:534)
at org.eclipse.jface.viewers.AbstractTreeViewer$1.run(AbstractTreeViewer.java:719)
at org.eclipse.swt.custom.BusyIndicator.showWhile(BusyIndicator.java:67)
at org.eclipse.jface.viewers.AbstractTreeViewer.createChildren(AbstractTreeViewer.java:696)
at org.eclipse.jface.viewers.TreeViewer.createChildren(TreeViewer.java:892)
at org.eclipse.jface.viewers.AbstractTreeViewer.internalExpandToLevel(AbstractTreeViewer.java:1486)
at org.eclipse.jface.viewers.AbstractTreeViewer.expandToLevel(AbstractTreeViewer.java:877)
at com.sap.tools.memory.ui.views.editors.ObjectsListPane$6.doubleClick(ObjectsListPane.java:743)
at org.eclipse.jface.viewers.StructuredViewer$1.run(StructuredViewer.java:796)
at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:37)
at org.eclipse.core.runtime.Platform.run(Platform.java:843)
at org.eclipse.ui.internal.JFaceUtil$1.run(JFaceUtil.java:44)
at org.eclipse.jface.util.SafeRunnable.run(SafeRunnable.java:149)
at org.eclipse.jface.viewers.StructuredViewer.fireDoubleClick(StructuredViewer.java:794)
at org.eclipse.jface.viewers.AbstractTreeViewer.handleDoubleSelect(AbstractTreeViewer.java:1216)
at org.eclipse.jface.viewers.StructuredViewer$4.widgetDefaultSelected(StructuredViewer.java:1158)
at org.eclipse.jface.util.OpenStrategy.fireDefaultSelectionEvent(OpenStrategy.java:223)
at org.eclipse.jface.util.OpenStrategy.access$0(OpenStrategy.java:220)
at org.eclipse.jface.util.OpenStrategy$1.handleEvent(OpenStrategy.java:281)
at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:66)
at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:928)
at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:3348)
at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:2968)
at org.eclipse.ui.internal.Workbench.runEventLoop(Workbench.java:1914)
at org.eclipse.ui.internal.Workbench.runUI(Workbench.java:1878)
at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:419)
at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:149)
at com.sap.tools.memory.ui.rcp.Application.run(Application.java:14)
at org.eclipse.core.internal.runtime.PlatformActivator$1.run(PlatformActivator.java:78)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:92)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:68)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:400)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:177)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.eclipse.core.launcher.Main.invokeFramework(Main.java:336)
at org.eclipse.core.launcher.Main.basicRun(Main.java:280)
at org.eclipse.core.launcher.Main.run(Main.java:977)
at org.eclipse.core.launcher.Main.main(Main.java:952)

Hi Scott,

We checked your exceptions and believe to have found the problem. Unfortunately, you haven't exposed any contact information, e.g. an e-mail address in your SDN account. We can't contact you to give you the patched plugin to retest your use case.

Can you please e-mail either to me (vedran.lerenc@sap.com) or anybody else on the Wiki page listed as contacts?

And for all others with problems: Feel free to e-mail us. We will see what we can do for you.

Thank you in advance,

Vedran

Greertings to the team

I downloaded the linux version and on OpenSuse 10.2 and the shell script complains :

~/bin/MemoryAnalyzer>Error occurred during initialization of VM
Incompatible initial and maximum heap sizes specified

----------

~/bin/MemoryAnalyzer>java -version
java version "1.5.0_13"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_13-b05)
Java HotSpot(TM) Client VM (build 1.5.0_13-b05, mixed mode, sharing)

Adding -Xmx bigger than the -Xms fixes the problem - I know it is not flexbie, but maybe default -Xmx will be little more user-friendly

Regards,

 (ex-deploy) Georgi

Hi Georgi,

Great to hear from you and thank you for your comment.

I just checked the scripts, but I couldn't find any -Xmx parameter. The scripts just have -Xms set. Can you elaborate a little bit more on the problem? Maybe I got you wrong?

Hope everything is fine in your new job?

Kind Regards,

Vedran

Hi Georgi,

it seems that the default value that is chosen for the -Xmx option is smaller than the value we specified for the -Xms option in the shell script:

java -Xms512m -jar plugins/org.eclipse.equinox.launcher_1.0.0.v20070606.jar &

Therefore we will add a value for the -Xmx option in our shell scripts and update our download packages.
We will set the value for this option to 1024m initially, but the value can be changed individually.

Kind Regards, Erwin

Hello, Team:

 This is a great tool and I can image that I will use it for a long time. It's much better comparing to jhat, gcviewer, and other tools that I used.

One question: I am chasing a OutOfMemory exception from PERM generation. While incrasing the perm size will likely to solve(or I should say hide) the problem, I want to get to the root of the issue, as I don't have the exception until a recent update of some software. In SAP Memory Analyzer, how can I view what are in perm generation, and their size?

Regards, Jeff

Hi Jeff,

Thank you for your comments! Unfortunately, HPROF binary heap dumps don't contain much valuable information from perm space. The only thing you get are class names and some few bits of information on the classes themselves like their inheritance hierarchy or their defining classloader, but that's it. Interned Strings are also available.

However, there is one query that might help you: Open Query Browser (fourth button on the tool bar) -> Core -> Duplicate Classes. This lists you all classes loaded twice (more precise: all classes with the same package and class name and their corresponding class loader). What also might help (a bit) is use grouping by class loader in class histogram or dominator tree. Usually perm space problems also manifest in heap.

But if it is just about big classes, then you have a problem. You have a problem fixing it, too, because you can't do much about big classes. What we found out, though: If you use lots of generators for a model/proxy-driven approach, then you might save a lot of memory by improving the generator templates (and factor out some often repeating code) or, if this suits your use case, deploy them only once instead of with every consumer.

Last but not least, try jmap. This tool is by Sun and may help you with your problem. For us it turned out to be not precise enough as the accounting of the memory that goes to the various perm space pools is pretty poor. Another group at SAP improved jmap for SAP JVMs and there the accounting of memory to a class is now much more useful.

If you even have a heap dump from a SAP Application Server you could use some special features within the Memory Analyzer, e.g. Leaking Loaders (again Open Query Browser -> SAP -> Leaking Loaders) gives you all class loaders that shouldn't be in memory anymore, but which are. We know about them because we know in our own application server where to look for the still registered class loaders. All other class loaders in heap are unregistered ones which don't go away, e.g. due to poor shutdown of the corresponding service or application. Unfortunately we can't tell that for an arbitrary heap dump, because this is application server specific.

I hope this was helpful to you,
Best regards,

Vedran

Thanks, Vedran. What I found out is that leaks in perm can be even tougher to pin point than the ones in heap. The leaks in heap can be identified even with hprof raw logs. For perm I only have -XX:+TraceClassLoading and Unloading flags. And the leaks can be caused by classloaders of the app server, which becomes quite tricky. Looks like I will have to figure out from perm class loading/unloading logs.

 Thanks! -Jeff

Vedran,

I am having difficulty on opening up a 2.3 G hprof dump file:

 (1). Using Solaris version of MemoryProfiler, tried switch -Xmx with different values from 2048m upto 3800m, but always hit error: "The file java_pid23778 could not be found" dialog. But the file is actually there. I tried other hprof with smaller size and it works fine.

 (2). On Windows XP, I am having problem giving MemoryAnalyzer a heap size larger than 2G. -Xmx 2048 will cause error: Could not create the Java Virtual Machine.

Any idea?

Kind Regards, Jeff

Hi Jeff,

1) Can you check if the user you use to read the file has enough permissions? I've had sometimes the problem that the heap dump is written only with read access for the user under which the application was started, and then I had to change the permissions before analyzing with another user.

2) On Windows XP you can have at maximum 2G for the process. In this 2G should fit the native heap needed for the VM itself, loaded dlls, JNI coding; then the java perm space, and the java heap space. From my experience you can get for -Xmx at maximum between 1200M and 1500M. The number differs from machine to machine, as it also depends on the fragmentation of the 2G by the different loaded dlls.

If the suggestions don't help, feel free to send me a mail (it is available in the Wiki) and we can continue to investigate.

Regards, Krum

Hello,

 I've been using the JavaOne 2007 version for a while now.  Is there an updated version available with the recent 1.1.1 features?  I don't use the tool as an IDE plug-in, just the standalone.

 Thanks for a really great tool,

-Ken

Hi Ken,

download the current version here. These are the New & Noteworthy with the new features.

Also, we now have a forum which can be used for questions and answers.

Kind regards,

-Andreas.

Just wanted to say thanks for producing such a great tool, looking forward to when it goes open source so that I can use it at work!

Hi,

 We run WAS 6.40 SP17 and EP 6.0 on HP-UX B11.23 (64B) with Java 1.4.2.15 and I'm wondering if the Memory Analyzer can be installed in our platform. I can only see the available intallers for Windows, Linux and Solaris.

Could you please advice me?

Regards,

HP.

Hi Nestor,

Short answer: Yes, transfer the dump to a supported platform and analyze it there.

Long answer:
The Memory Analyzer differentiates between "runtime" and "analysis" platform. The runtime platform is the platform where the HPROF file is generated. This must be Sun, SAP and HP JDK/JVM from version 1.4.2_12 or 5.0_7 or 6.0 upwards.

The analysis platform is where the tool runs. The tools needs (a) a 5.0 JRE and (b) SWT platform binaries. The Memory Analyzer then reads all dumps generated on the supported runtime platforms.

As you create the heap dump on a 64Bit machine, it might need a 64bit analysis platform. This really depends on the number of objects. If you have few but big objects, then 32bit can be enough.

Also, we now have a forum which can be used for questions and answers.

Kind regards,

-Andreas.

Hi Experts

Thanks for providing such a powerful  tool !

Anwyay, i'm getting an error message while trying to run  the "Leak suspect" report. Indeed i generated a heapdump on my Sandbox Netweaver server then i retrieved it to my PC. I'm able to display some informations (so i'm assuming my heapdump is correct) in Memory Analyzer but the error came with "Leak suspect" report.

Here is the whole error message :

eclipse.buildId=unknown
java.version=1.6.0_11
java.vendor=Sun Microsystems Inc.
BootLoader constants: OS=win32, ARCH=x86, WS=win32, NL=fr_FR
Command-line arguments:  -os win32 -ws win32 -arch x86

Error
Fri Jan 29 15:37:23 CET 2010
An internal error occurred during: "default_report  org.eclipse.mat.api:suspects".

java.lang.AbstractMethodError: com.sap.tools.memory.netweaver.csn.CSNComponentExtractor.resolveByClassLoader(Lorg/eclipse/mat/snapshot/model/IClassLoader;Lorg/eclipse/mat/util/IProgressListener;)Ljava/lang/String;
at org.eclipse.mat.inspections.LeakHunterQuery.getTroubleTicketMapping(LeakHunterQuery.java:558)
at org.eclipse.mat.inspections.LeakHunterQuery.appendTroubleTicketInformation(LeakHunterQuery.java:701)
at org.eclipse.mat.inspections.LeakHunterQuery.getLeakDescriptionSingleObject(LeakHunterQuery.java:382)
at org.eclipse.mat.inspections.LeakHunterQuery.getLeakSuspectDescription(LeakHunterQuery.java:215)
at org.eclipse.mat.inspections.LeakHunterQuery.execute(LeakHunterQuery.java:148)
at org.eclipse.mat.query.registry.ArgumentSet.execute(ArgumentSet.java:129)
at org.eclipse.mat.query.registry.CommandLine.execute(CommandLine.java:93)
at org.eclipse.mat.report.internal.QueryPart.execute(QueryPart.java:96)
at org.eclipse.mat.report.internal.SectionPart.execute(SectionPart.java:53)
at org.eclipse.mat.report.TestSuite.execute(TestSuite.java:127)
at org.eclipse.mat.report.internal.RunRegisterdReport.execute(RunRegisterdReport.java:50)
at org.eclipse.mat.query.registry.ArgumentSet.execute(ArgumentSet.java:129)
at org.eclipse.mat.ui.QueryExecution$ExecutionJob.run(QueryExecution.java:174)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55)

Could you please give me some advice ?

Thanks

François