A perceived memory leak may not actually be a memory leak. No, we’re not trying to confuse you, or start an existential debate! The fact of the matter is that there are applications that may be perceived to be leaking that are actually consuming memory by design. Assuming that you actually do have a leak, capturing Performance Monitor Data may help you identify what is leaking, but generally does not provide more information beyond that. The key piece of data in definitively identifying the memory leak within an application is the thread stack. The thread stack is a list of functions. Trying to capture the exact thread stack that is leaking is difficult because threads execute so fast that knowing exactly when to dump the process is not an easy task.
This is where UMDH.EXE comes into play. UMDH stands for User Mode Dump Heap. UMDH is a lightweight standalone executable that does not require a separate installation routine beyond the installation of the Debugging Tools for Windows. Using UMDH you can find the exact set of function calls responsible for the memory consumption. UMDH make use of the Global Flags (GFLAGS) tool settings. Below are the steps for using UMDH:
The first thing you need to do is enable a setting in GFLAGS called “Create user mode stack trace database” for the process that you are investigating. This setting creates a registry value which is read when the process starts up that allows the system to keep track of the functions allocating memory inside the process. You can enable this value either via the GUI or via the command-line. When enabling this value through the GUI, remember to switch to the Image File tab (as shown below) – in this case, I am selecting NOTEPAD.EXE as the process to watch:
The syntax to set this value from the command line is: gflags.exe -i notepad.exe +ust . If the process is already running, you will need to restart the process for this change to take effect.
In addition to setting the Global Flags setting for the process, you will also need to configure the symbol path. You can configure the Symbol Path by setting an environment variable within the properties of “My Computer”. The environment variable name is _NT_SYMBOL_PATH . To set up a symbol path to point to the Microsoft public symbol server, set the variable as follows: srv*c:\symbols*http://msdl.microsoft.com/download/symbols . This creates a local symbol store at C:\Symbols that is populated from the public symbol server. Having this symbol path set allows UMDH to resolve binary information to friendly function names.
Once we have our configuration settings in place, we can take our UMDH snapshots. Taking a snapshot is a straightforward process – the one thing you will need is the Process ID (which you can find by using Task Manager). The syntax is as follows: umdh –p:<PID> –f:<filename>.txt . The – p switch specifies the Process ID and the – f switch specifies the file name for the snapshot file. After you have allowed some time to pass (and the process has leaked some memory), you can take your second snapshot using the same syntax as above. When taking snapshots, it is important to remember that each output file is a discrete entity – data is not appended to the end of each file. So if you set up a batch file to run every five minutes, ensure that the output file name is different for each snapshot.
Once you have at least two snapshots you can use UMDH to compare the two files and create a more useful output file using the following command: umdh -v<filename1>.txt <filename2>.txt > comparefiles.txt .
Assuming that you have set up all the configuration pieces correctly, the output in the output file that you used to capture the comparison data should have text along the lines of what is below in it. In my example I used a Windows Server 2008 x64 machine and captured two quick snapshots of Notepad.exe:
DBGHELP: NOTEPAD - public symbols
c:\symbols\notepad.pdb\DA66AEF719A14FF9902CB3D8EDB248E71\notepad.pdb
DBGHELP: ntdll - public symbols
c:\symbols\ntdll.pdb\C43EA3129CCD4F36AEE8C7BADA1D24C32\ntdll.pdb
DBGHELP: kernel32 - public symbols
c:\symbols\kernel32.pdb\B711C92DF84F44F389CA44A27E40E0662\kernel32.pdb
DBGHELP: ADVAPI32 - public symbols
c:\symbols\advapi32.pdb\2DBF14CE6CE345F98330B64E4588404F2\advapi32.pdb
DBGHELP: RPCRT4 - public symbols
c:\symbols\rpcrt4.pdb\F67F7476256149E38518882D9FB205912\rpcrt4.pdb
DBGHELP: GDI32 - public symbols
c:\symbols\gdi32.pdb\B62EE5B40BF34CFFA631BD8EA263A4F62\gdi32.pdb
DBGHELP: USER32 - public symbols
c:\symbols\user32.pdb\C60349DEF05F4E4AACA4327BB0351F242\user32.pdb
DBGHELP: msvcrt - public symbols
c:\symbols\msvcrt.pdb\6E1182A1FD174EED8DB7F08E27A65F8E2\msvcrt.pdb
DBGHELP: COMDLG32 - public symbols
c:\symbols\comdlg32.pdb\F0E4114C89A242E787FA1E3FB486B05B2\comdlg32.pdb
DBGHELP: SHLWAPI - public symbols
c:\symbols\shlwapi.pdb\74EACBA4506C4720B746D5E41FC230262\shlwapi.pdb
DBGHELP: COMCTL32 - public symbols
c:\symbols\comctl32.pdb\C2284C28FC73411FB2821A0C46DC2B171\comctl32.pdb
DBGHELP: SHELL32 - public symbols
c:\symbols\shell32.pdb\F9FF47320B1942B49B3BE594C72DC84A2\shell32.pdb
DBGHELP: WINSPOOL - public symbols
c:\symbols\winspool.pdb\A5812C87A42B4B409B4F3783389438DA1\winspool.pdb
DBGHELP: ole32 - public symbols
c:\symbols\ole32.pdb\6F224BB1EB074553BCB6C99865FD6B832\ole32.pdb
DBGHELP: OLEAUT32 - public symbols
c:\symbols\oleaut32.pdb\4B08C8B9532D4F88926235FD31E73B902\oleaut32.pdb
DBGHELP: IMM32 - public symbols
c:\symbols\imm32.pdb\64260C28FC674491A937B23C381011DC2\imm32.pdb
DBGHELP: MSCTF - public symbols
c:\symbols\msctf.pdb\41F8309E3E3F4817951BA62D85F4A5E12\msctf.pdb
DBGHELP: LPK - public symbols
c:\symbols\lpk.pdb\595ABFF66E154C36972782446C646DF72\lpk.pdb
DBGHELP: USP10 - public symbols
c:\symbols\usp10.pdb\3A057D0C5E2C473AB6D891ACA7CA77452\usp10.pdb
DBGHELP: uxtheme - public symbols
c:\symbols\UxTheme.pdb\957EDB1FC41740F2B701D7E762173EB32\UxTheme.pdb
DBGHELP: RTSUltraMonHook - export symbols
DBGHELP: RTSUltraMonHookRes - no symbols loaded
DBGHELP: MSIMG32 - public symbols
c:\symbols\msimg32.pdb\07A859601B38486C81ACFEC163CDD5E62\msimg32.pdb
DBGHELP: CLBCatQ - public symbols
c:\symbols\CLBCatQ.pdb\1A9CD492EE354773AE3946B3C83009BE2\CLBCatQ.pdb
//
// Each log entry has the following syntax:
//
// + BYTES_DELTA (NEW_BYTES - OLD_BYTES) NEW_COUNT allocs BackTrace TRACEID
// + COUNT_DELTA (NEW_COUNT - OLD_COUNT) BackTrace TRACEID allocations
// ... stack trace ...
//
// where:
//
// BYTES_DELTA - increase in bytes between before and after log
// NEW_BYTES - bytes in after log
// OLD_BYTES - bytes in before log
// COUNT_DELTA - increase in allocations between before and after log
// NEW_COUNT - number of allocations in after log
// OLD_COUNT - number of allocations in before log
// TRACEID - decimal index of the stack trace in the trace database
// (can be used to search for allocation instances in the original
// UMDH logs).
//
As you can see from the output, any Windows binaries for which public symbols (SHELL32.DLL, NTDLL.DLL etc) are available has a path to the symbols listed. For any binaries for which symbols are not available from my source symbol server, there is no path listed.
With that, we’re going to bring this post to a close. In a future post, we'll go over troubleshooting a memory leak using UMDH.EXE.
- Mike Morales
| Share this post : |
|
|
|
|
|