High Memory Usage or Memory Leaks in Web Applications: Understanding and Data Collections

Introduction

High memory usage or memory leaks is a common challenge for the web applications. The unanticipated memory consumption can lead to performance bottlenecks, system crashes, and degraded user experiences. In this article, we will explore the concept of high memory usage, how to identify it, various types of high memory issues and how to collect logs for analysis.

Identifying high memory

The simplest way to identify high memory is to check the Private Bytes. Private Bytes indicates the amount of private committed memory being used by the process. Making it the key counter to rely on when determining if high memory consumption is happening for the application.

IIS Manager: You can check the Private Bytes form the IIS Manager Worker Process module. The private bytes displayed here is in KB.

Task Manager: You can get the same information from Task Manager Details tab. Find the worker process(w3wp.exe) that match the user name for that application pool. You need to look for Memory (active private working set). You will get the private bytes consumed by the application in KB.

Performance Monitor: Performance Monitor is a great tool to verify high memory usage. 

• Open Performance Monitor
• Click Add
• Expand Process
• Select Private Bytes
• And choose the worker process (w3wp).

Also add the following counters -

• Virtual Bytes, you can find the Virtual Bytes under Process tree.
• #Bytes in all Heap, you will find the #Bytes in all Heap under .NET CLR Memory.

These counters will help you to identify the memory leaks. And it will also tell you whether a leak is a native leak or a managed leak.

Native Memory Leaks: If the private bytes counter is increasing but the .NET bytes in all heaps counter remains constant. This indicates a native memory leak.

Managed Memory Leaks: If the private bytes counter and the .NET bytes in all heaps counter are increasing at the same rate (the difference between the two remains constant).

indicates a managed memory leak.

Log Collection

Now that you're familiar with how to confirm high memory and use counters to distinguish between native and managed memory leaks, the next step is to correctly collect logs. In most cases, a full user dump is sufficient for analyzing high memory or memory leaks. However, the process for collecting dumps differs between native and managed leaks.

Native Memory Leaks

• Download and install DebugDiag tool from this official download link - Download Debug Diagnostic Tool v2 Update 3 from Official Microsoft Download Center
• Open DebugDiag 2 Collection.
• Go to Processes tab
• Select the worker process (w3wp.exe).
• Right click and select Monitor For Leaks
• And click Create Full Userdump

This will generate a full user dump, and you should create three of them for proper analysis and comparison.

You can automate this process if the issue is intermittent or you don’t want to monitor it. 

• Open DebugDiag 2 Collection.
• Click on Add Rule
• Select Native (non-.NET Memory and handle Leak)
• Click Next
• Select the worker process (w3wp.exe)
• Click Configure and set provide the parameters as in below screenshot.

The above parameters will create the first memory dump of 800 MB. Two more dumps will be created at the increments specified - i.e. 1000 (800+ 200) and 1200 MB. Then the final memory dump will be created 15 minutes after tracking.

Managed Memory Leaks 

• Download and install DebugDiag tool from this official download link - Download Debug Diagnostic Tool v2 Update 3 from Official Microsoft Download Center
• Open DebugDiag 2 Collection.
• Go to Processes tab
• Select the worker process (w3wp.exe) for the application.
• Right click and click Create Userdump Series

• Select and set the below options,  do not click "Save & Close" at this point.
• Wait for memory consumption to raise to decided level (70 to 80%).
• Click Save & Close

You can automate this process using the ProcDump utility. 

• Download Procdump.exe from thi official download link - ProcDump - Sysinternals | Microsoft Learn
• Extract the zip files into a folder of your choice.
• Open command prompt with administrator privilege and navigate to the folder.
• Execute the below command.

procdump.exe -s 30 -m 1000 -ma -n 3 <PID>

-n number of memory dumps

-m Memory commit threshold in MB at which the dumps will be created

-s would indicate the number of consecutive seconds where memory consumption was >= threshold specified with -m

PID is the process id for the worker process.

.NET Core applications

If the app in question is .NET Core and hosted on IIS in-process mode, then this above option applies as is. But if the app is hosted on IIS as out-of-proc mode then the action plan should be modified so that the dotnet process(dotnet.exe unless otherwise specified) is investigated instead of w3wp.exe. Same thing applies self-hosted .NET Core applications.

Conclusion

High memory usage or memory leaks is a complex issue with many potential causes, it can be native or managed. It is very important to isolate the type of memory leaks and carefully capture the logs. You can analyze the dumps using tools like WinDbg or DebugDiag2 Analysis. If you want us to do that, please contact us with a case and we will do it for you.