Memory dumps for hang or performance issues, "manual" approach

[Pingulea]

You should be using this method when the slow HTTP processing can be easily reproducible, when you can easily predict that a request processing would be slow.

An alternative to the approach below is to trigger a memory dump collection with a FREB rule, according to instructions in this article: http://linqto.me/freb-dump.
The only amendment to that article: I would take 2 or 3 dumps one after another, some 10 seconds apart; so, I would use customActionParams="-accepteula -ma -n 3 -s 10 %1% C:\MyDumps".

Both ProcDump and DebugDiag can attach to a process and collect memory dumps from it. DebugDiag is more flexible, ProcDump has almost no footprint. 

Using the command-line ProcDump

ProcDump does not require installation. Needs to be run from an administrative console. But ProcDump is not as easily configurable as DebugDiag…

1. Download the tool and copy it on a disk folder, for example D:\Temp-Dumps\
   https://docs.microsoft.com/en-us/sysinternals/downloads/procdump
    
2. Open an administrative console, from where to run commands.
   Then navigate to where the ProcDump tool was placed (D:\Temp-Dumps\).
    
3. Find the process ID, the PID, of the IIS w3wp.exe worker process executing your application.
   From the administrative console, you can use AppCmd IIS tool to list processes for application pools:

   D:\Temp-Dumps> C:\Windows\System32\InetSrv\appcmd.exe list wp

    
4. In the administrative console, prepare the following command to collect dumps (but don't hit enter yet).

   D:\Temp-Dumps> procdump.exe -accepteula -ma -n 5 -s 10 [PID]

   Replace [PID] with the actual Process ID integer number identified at the step 2.
   Please make sure that there is enough disk space on the drive where dumps are collected. Each process dump will take space in the disk approximately the same size the process uses in memory (column Commit Size in Task Manager). For example, if the process’ memory usage is ~1 GB, then the size of a dump file will be around 1 GB.
    
5. Start reproducing the problem: issue a slow-running request from the client (browser) to the investigated IIS application.
   Or simply wait or make requests to the IIS/Asp.Net app until the slowness manifests.
   Then, as fast as possible…
    
6. Hit enter on the above ProcDump.exe command (procdump.exe -accepteula -ma -n 5 -s 10 [PID])
    
7. Please wait until all the dumps are written and then compress the files before uploading the dumps to the workspace.

Using the UI tool DebugDiag, Debug Diagnostics Collection

DebugDiag requires installation, but it is able to determine itself the whatever process instance - PID - happens to execute for an application pool at some point in time; even when that process may occasionally crash, hence restart with different PID.

#1.

Download and install the Debug Diagnostic tool on IIS machine:

https://www.microsoft.com/en-us/download/details.aspx?id=49924 v2.2 (if 32-bit system)
https://www.microsoft.com/en-us/download/details.aspx?id=103453 v2.3.2 (only supports 64-bit OS)

#2.

Open Debug Diagnostic Collection.
If a “rule wizard” appears, please click Cancel.
 

#3. 

You may want to change the default location where the memory dumps are written.
By default, these are placed under C:\Program Files\DebugDiag\Logs\Misc
Change from Menu > Tools > Options And Settings, then modify Manual Userdump Save Folder:

Please make sure that there is enough disk space on the drive where dumps are collected. Each process dump will take space in the disk approximately the same size the process uses in memory (column Commit Size in Task Manager). For example, if the w3wp.exe process memory usage is ~2 GB, then the size of each dump file will be around 2 GB.
Please do not choose a disk in network/UNC; choose a local disk.

#4.

Go to Processes tab. Sort the processes by Process Name and find the w3wp.exe process.
If there are more than one application pool running at the same time you may see more than one w3wp.exe.
In that case, you may differentiate by Process Identity or by checking the Web Application Pool Name column.
 

#5.

Right click on the process and choose “Create Userdump Series…” and then…
 

#6.

Collect the dump series with the following settings:

1. Generate a Userdump every 10 seconds.
2. Start the timer when writing the dump file completes.
3. Stop after generating 5 userdumps.
4. Collect Full Userdumps.
5. As soon as you click Save & Close, you will see the following pop-up box and the dump collection will be started immediately:

#7.

Please wait until all the dumps are written on disk at the configured location.

Archive each dump file in its own ZIP and prepare to hand over to the support engineer; upload in a secure file transfer space.

Remember my article about how exceptions are handled and how to collect memory dumps to study them. We can double check if a crash occurred or not: read about w3wp.exe crashes.

Aside: Just in case you are wondering what I use to capture screenshots for illustrating my articles, check out this little ShareX application in Windows Store.