IIS Crash Dump Collection

A Comprehensive Guide to Diagnosing IIS Crashes

Table of Contents

• What is Crash?
• What is Exception?
• How to Verify Same?
• What Logs Can Be Collected?
• What is Dump?
• Difference Between Crash Dump and Hang Dump
• Tools That Can Be Used to Collect Crash Dump
• Steps to Collect
• Summary

What is Crash?

A crash in the context of IIS (Internet Information Services) refers to an unexpected termination of the IIS process, causing the web server to stop functioning properly. Crashes can be triggered by several factors, including software bugs, hardware failures, or configuration issues. When IIS crashes, it can lead to downtime, affecting the availability of web applications and services hosted on the server.

What is Exception?

An exception is an error condition that occurs during the execution of a program. In IIS, exceptions can occur due to various reasons, such as invalid input, resource unavailability, or bugs in the application code. Exceptions can be caught and handled gracefully by the application to prevent a crash, but unhandled exceptions often lead to process termination and crashes.

How to Verify Same?

To verify whether an IIS crash has occurred, you can check the following:

• Error messages in the Event Viewer under Windows Logs > Application.
• Application logs generated by your web application.
• HTTP error codes and responses served to clients.
• Service status of IIS and related services.

What Logs Can Be Collected?

When diagnosing IIS crashes, it is essential to collect various logs that provide insights into the issue. These logs include:

• Event Viewer logs (Application, System, and IIS-related logs).
• IIS log : "C:\inetpub\logs\LogFiles" default directory.
• Application-specific logs generated by the web application.
• Dump collection.

What is Dump?

A dump file is a snapshot of an application's memory at a specific point in time, usually when a crash or exception occurs. It contains detailed information about the application's state, including the call stack, heap, and active threads. Dumps are invaluable for debugging and diagnosing the root cause of crashes.

Difference Between Crash Dump and Hang Dump

A crash dump is created when a system or application crashes due to an unhandled exception or critical error, such as a memory access violation, divide-by-zero error, or stack overflow. In contrast, a hang dump is generated when a system or application becomes unresponsive (hangs) without crashing. Crash dumps are used to diagnose unexpected crashes, while hang dumps are used to debug issues related to unresponsiveness. The main differences between crash dumps and hang dumps lie in their triggers, purposes, and content. Here's a comparison:

Tools That Can Be Used to Collect Crash Dump

Several tools can be used to collect crash dumps for IIS, including:

• DebugDiag: A tool designed to help troubleshoot application crashes, hangs, and memory leaks. It can capture crash dumps and analyze them.
  
  
• ProcDump: A command-line utility that allows you to monitor and generate crash dumps for processes based on specific triggers, such as unhandled exceptions.
  
  
• WER: Windows Error Reporting (WER) is a built-in feature in Windows that collects crash dumps and error data when an application or system encounters a critical error. It can automatically generate crash dumps and send diagnostic information to Microsoft or save them locally for debugging. WER simplifies crash analysis by capturing relevant details such as faulting modules, exception codes, and memory snapshots.

Steps to Collect

Collecting crash dumps for IIS involves several steps:

Using DebugDiag

Steps to capture Debug Diag Dumps to capture second chance exception  :

1. Debug Diagnostic 2.0 can be download at the following URL: 
Debug Diag Download

2. Execute DebugDiag 2.0 Collection on the IIS server, the wizard "Select Rule Type" loads

3. Select "Crash" and then click on "Next >"

 

 

4. In the window "Select Target Type"

5. Check "A specific IIS web application pool", then click on "Next >"

 

[This will have for effect to only generate a dump on the W3WP.exe process corresponding to your Application Pool. If you want to monitor every processes owned by IIS, you can select "All IIS/COM+ related processes". You can also select "A specific process" to monitor every instances of a process or just a unique one (a specific process with a specific PID)

The drawback of "A specific process" for a unique instance is you need to recreate the rule each time the process is restarted as his PID would have changed]

 

 

6.Select the Application Pool facing the crash issue InternetBankingHost not DefaultppPool, then click on "Next >"

 

 

7.Add the breakpoint as you can see in the below screenshot

[This allows you to generate a dump as soon as the command "TerminateProcess" is sent in the process without waiting for a complete stop

In addition, if a problem is detected by WAS in the process, it will stop it with this command, so without this breakpoint, no dump will be generated while the process is facing a crash]

 

 

8. Then click on "Next >"

 

 

 

 

9. Choose a name and a location for dumps file, then click on "Next >"

10. Setting those dumps on another disk than C:\ is possible

11. Please check you have enough available space disk as the dump size will be equal to the memory consumed by the process you are monitoring

 

 

12. In the window "Rule Completed", select "Activate the rule now", and then click on "Finish"

13. The rule is now configured and ready to generate dumps

 

 

 

Using ProcDump

1. Download Procdump to your local machine from 
   Procdump Download
2. Open command prompt in admin mode
3. Browse to the location where procdump is present
4. Run the application and wait to send the message (wait before you reproduce the error)

 

Run the command :

Procdump.exe -e -t -ma <PID>
Procdump.exe -e -t -ma w3wp

 

• -ma : Write a 'Full' dump file. Includes all memory (Image, Mapped and Private) and metadata (Process, Thread, Module, Handle, Address Space, etc.).

• -t : Write a dump when the process terminates.

• PID : Process ID

• -e : Write a dump when the process encounters an unhandled exception.

 

WER Dumps

To capture more information about crashes and application exceptions, we will configure WER (Windows Error Reporting) to write full dumps of crashing applications:

 

1. Create a folder C:\dumps
2. Copy the following lines to Notepad and save it as a .REG file.

 

Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\Windows Error Reporting\LocalDumps]
"DumpFolder"=hex(2):63,00,3a,00,5c,00,64,00,75,00,6d,00,70,00,73,00,00,00
"DumpCount"=dword:0000000a
"DumpType"=dword:00000002

 

3. Optional:   Backup the contents of the keys listed below in case you want to restore them later:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\Windows Error Reporting\LocalDumps

 

4.  Double-click the .REG file to import the values into the registry.

 

If you wish to change the path where dumps are stored, you can edit the following key to reflect the preferred location:   

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\Windows Error Reporting\LocalDumps\DumpFolder

 

Summary

In conclusion, IIS crashes can significantly impact the availability and performance of web applications. Understanding the nature of crashes and exceptions, verifying their occurrence, and collecting relevant logs and dumps are critical steps in diagnosing and resolving these issues.

 

Utilizing tools like DebugDiag, ProcDump, and WinDbg can simplify the process of capturing crash dumps and analyzing them. By following the outlined steps, you can effectively collect and interpret crash dumps to identify and fix the root causes of IIS crashes, ensuring a stable and reliable web server environment.