Home
%3CLINGO-SUB%20id%3D%22lingo-sub-482084%22%20slang%3D%22en-US%22%3EIIS%20application%20pool%20may%20be%20terminated%20when%20capturing%20a%20manual%20memory%20dump%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-482084%22%20slang%3D%22en-US%22%3E%3CP%3EConsider%20the%20following%20scenario%3A%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CUL%3E%0A%3CLI%3EYou%20are%20running%20a%20web%20application%20on%20IIS%20and%20you%20need%20to%20collect%20a%20memory%20dump%20of%20the%20application%20pool%20process%20(w3wp.exe)%2C%20e.g.%3A%20when%20trying%20to%20troubleshoot%20a%20performance%20issue.%3C%2FLI%3E%0A%3CLI%3Ew3wp.exe%20consumes%20large%20memory%20and%20it%20takes%20long%20time%20to%20write%20the%20dump%20file.%3C%2FLI%3E%0A%3CLI%3EWhen%20you%20try%20to%20collect%20a%20memory%20dump%20of%20the%20worker%20process%2C%20it%20is%20terminated%20and%20the%20memory%20dump%20is%20corrupted.%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3CP%3EAt%20the%20same%20time%20you%20may%20see%20the%20following%20event%20logged%20in%20the%20system%20event%20logs%3A%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CBLOCKQUOTE%3E%0A%3CP%3ELog%20Name%3A%20System%3CBR%20%2F%3ESource%3A%20Microsoft-Windows-WAS%3CBR%20%2F%3EDate%3A%20%3CDATE%20of%3D%22%22%20the%3D%22%22%20issue%3D%22%22%3E%3CBR%20%2F%3EEvent%20ID%3A%205010%3CBR%20%2F%3ETask%20Category%3A%20None%3CBR%20%2F%3ELevel%3A%20Warning%3CBR%20%2F%3EKeywords%3A%20Classic%3CBR%20%2F%3EUser%3A%20N%2FA%3CBR%20%2F%3EComputer%3A%20%3CNAME%20of%3D%22%22%20the%3D%22%22%20computer%3D%22%22%3E%3CBR%20%2F%3EDescription%3A%3CBR%20%2F%3EA%20process%20serving%20application%20pool%20'AppPool%20name'%20failed%20to%20respond%20to%20a%20ping.%20The%20process%20id%20was%20'%3CPID%3E'.%3C%2FPID%3E%3C%2FNAME%3E%3C%2FDATE%3E%3C%2FP%3E%0A%3C%2FBLOCKQUOTE%3E%0A%3CP%3EThis%20may%20be%20happening%20due%20to%20the%20%E2%80%9Chealth%20monitoring%E2%80%9D%20configuration%20of%20the%20application%20pool.%20Let's%20see%20what%20happens.%20The%20following%20screenshot%20shows%20the%20default%20settings%20for%20health%20monitoring%3A%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20style%3D%22width%3A%20563px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Fgxcuf89792.i.lithium.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F110173i440CC07DCE1EFA5A%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20alt%3D%22app_pool_advanced_settings.png%22%20title%3D%22app_pool_advanced_settings.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EThe%20following%20happens%20with%20the%20default%20configuration%20seen%20in%20the%20screenshot%20above%3A%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CUL%3E%0A%3CLI%3EWAS%20monitors%20the%20application%20pool%20by%20%22pinging%22%20it%20every%2030%20seconds%20(%3CSTRONG%3EPing%20Period%3C%2FSTRONG%3E%20configuration).%3C%2FLI%3E%0A%3CLI%3EIf%20the%20process%20does%20not%20respond%20in%2090%20seconds%20(%3CSTRONG%3EPing%20Maximum%20Response%20Time%3C%2FSTRONG%3E)%20then%20WAS%20will%20terminate%20the%20process%20considering%20that%20it%20is%20unhealthy.%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3CP%3ERemember%20that%20a%20process%20(actually%20its%20threads)%20is%20suspended%20when%20a%20dump%20is%20written%20to%20the%20disk%2C%20which%20means%20that%20it%20will%20not%20respond%20to%20WAS%E2%80%99%20ping%20requests%20until%20the%20dump%20file%20is%20written.%20If%20it%20takes%20longer%20time%20to%20write%20the%20dump%20file%20than%20the%20%22Ping%20Maximum%20Response%20Time%22%2C%20then%20WAS%20will%20terminate%20the%20process.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EIf%20you%20see%20the%20similar%20behavior%20and%20you%20need%20to%20collect%20a%20useful%20dump%20then%20you%20may%20need%20to%20increase%20the%20ping%20timeout%20or%20set%20%E2%80%9Cping%20enabled%E2%80%9D%20to%20false%20until%20you%20collect%20the%20memory%20dump%20(which%20is%20not%20recommended).%26nbsp%3B%3CSTRONG%3EPlease%20note%20that%20these%20changes%20will%20cause%20application%20pool%20recycle.%3C%2FSTRONG%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EThis%20story%20actually%20explains%20why%20it%20is%20important%20to%20capture%20the%20memory%20dumps%20in%20a%20local%20folder%20instead%20of%20a%20network%20share.%20The%20dump%20file%20should%20be%20written%20as%20fast%20as%20possible%20so%20the%20process%20is%20not%20suspended%20for%20a%20long%20time.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EHappy%20troubleshooting...%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E--%3C%2FP%3E%0A%3CP%3EAMB%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-TEASER%20id%3D%22lingo-teaser-482084%22%20slang%3D%22en-US%22%3E%3CP%3EHave%20you%20ever%20experienced%20a%20process%20termination%20when%20you%20try%20to%20capture%20a%20manual%20memory%20dump%20of%20an%20IIS%20worker%20process%3F%20It%20might%20be%20because%20of%20the%20WAS%20service%20was%20trying%20to%20keep%20the%20worker%20process%20healthy.%20Confused%3F%20Please%20continue%20reading%20to%20find%20out%20why%20before%20accusing%20the%20WAS.%3C%2FP%3E%3C%2FLINGO-TEASER%3E
Microsoft

Consider the following scenario:

 

  • You are running a web application on IIS and you need to collect a memory dump of the application pool process (w3wp.exe), e.g.: when trying to troubleshoot a performance issue.
  • w3wp.exe consumes large memory and it takes long time to write the dump file.
  • When you try to collect a memory dump of the worker process, it is terminated and the memory dump is corrupted.

At the same time you may see the following event logged in the system event logs:

 

Log Name: System
Source: Microsoft-Windows-WAS
Date: <date of the issue>
Event ID: 5010
Task Category: None
Level: Warning
Keywords: Classic
User: N/A
Computer: <name of the computer>
Description:
A process serving application pool 'AppPool name' failed to respond to a ping. The process id was '<PID>'.

This may be happening due to the “health monitoring” configuration of the application pool. Let's see what happens. The following screenshot shows the default settings for health monitoring:

 

app_pool_advanced_settings.png

 

The following happens with the default configuration seen in the screenshot above:

 

  • WAS monitors the application pool by "pinging" it every 30 seconds (Ping Period configuration).
  • If the process does not respond in 90 seconds (Ping Maximum Response Time) then WAS will terminate the process considering that it is unhealthy.

Remember that a process (actually its threads) is suspended when a dump is written to the disk, which means that it will not respond to WAS’ ping requests until the dump file is written. If it takes longer time to write the dump file than the "Ping Maximum Response Time", then WAS will terminate the process.

 

If you see the similar behavior and you need to collect a useful dump then you may need to increase the ping timeout or set “ping enabled” to false until you collect the memory dump (which is not recommended). Please note that these changes will cause application pool recycle.

 

This story actually explains why it is important to capture the memory dumps in a local folder instead of a network share. The dump file should be written as fast as possible so the process is not suspended for a long time.

 

Happy troubleshooting...

 

--

AMB