Blog Post

Apps on Azure Blog
3 MIN READ

WCF Based Web App Performance Issue Caused by Hitting MaxConcurrentSessions Limit of WCF

Weizhen_Sun's avatar
Weizhen_Sun
Icon for Microsoft rankMicrosoft
May 04, 2022

A .NET web app hosted on Azure App Service can be built on top of the Windows Communication Foundation (WCF) framework. We've seen a kind of performance/slow response/timeout issue of such web app, which is caused by the MaxConcurrentSessions limit of WCF is hit.

 

The symptom might include that after restarting the web app for a few hours, a percentage of the total requests (e.g. 1/3) get no response from the web server and timeout after 230 seconds. 230 seconds is the Azure default timeout value. The web app's memory usage may be observed to grow continuously since the issue starts. 

 

The default MAX concurrent session setting of WCF server is 5000. If the WCF clients don’t close channels  properly, a lot of sessions which have been completed will not be able to get closed at the WCF server side. This will result in the MaxConcurrentSessions limit of WCF is always hit. Future requests might get 230 seconds timeout results from the WCF server. 

 

We could configure App Service's Auto-Heal rule to collect memory dump for the w3wp instances once the issue happens. You could find Auto-Heal tool from "Diagnose and Solve Problems" on the Azure App Service portal.  Then we can analyze the memory dump in the following way. 

 

  • Download the NetExt extension from here Releases · rodneyviana/netext · GitHub
  • Copy the NetExt.dll file to WinDbg installation folder, e.g. C:\Program Files\Debugging Tools for Windows (x64)\
  • Run WinDbg, open the memory dump file, load the NetExt dll by 0:000> .load NetExt
  • Run 0:000> !windex
  • Run 0:000> !wservice

When issue happens, you may see result like blow 

 

You can see the current session count is 5000 and the Max session count is also 5000. 

You can run !dumpheap -type System.ServiceModel.Security.SecuritySessionServerSettings to check Wcf objects. 

 

SecuritySessionServerSettings is the important object of the WCF session/connection. You can view the activeSessions object within the SecuritySessionServerSettings object. The "Count" value of the activeSessions object contains the number of the active WCF sessions. It also equals to the number of the System.ServiceModel.Security.SecuritySessionServerSettings+SecurityReplySessionChannel objects. 

 

A comparison of two memory dumps of w3wp when issue happened and not happened.

 

Dump 1# - Working (Issue not replicated)

Time      

Tue Nov  9 08:34:53.000 2021 (UTC + 8:00)

WCF sessions

0:000> !wservice

Sessions/Max   

0n2230/0n5000   

    

Number  and Size of objects related with WCF sessions

MT    Count    TotalSizeClass Name

 

00007ffdb2d18940     2238       304368   System.ServiceModel.Channels.TransactionChannelListener`1+TransactionReplySessionChannel[[System.ServiceModel.Channels.IReplySessionChannel, System.ServiceModel]]

 

00007ffdb2912208     8988       2516640 System.ServiceModel.Channels.ServiceChannel

 

00007ffdb2cf8188      2238       572928   System.ServiceModel.Security.SecuritySessionServerSettings+SecurityReplySessionChannel

 

Number of Active Sessions

 

 

 

 

 

Dump 2# - Issue (replicated)

Time

Tue Nov  9 18:03:55.000 2021 (UTC + 8:00)

WCF sessions

0:000> !wservice

Sessions/Max         

0n5000/0n5000         

Number  and Size of objects related with WCF sessions

MT    Count    TotalSizeClass Name

 

00007ffdb2d18940     5001        680136     System.ServiceModel.Channels.TransactionChannelListener`1+TransactionReplySessionChannel[[System.ServiceModel.Channels.IReplySessionChannel, System.ServiceModel]]

 

00007ffdb2912208     11530      3228400   System.ServiceModel.Channels.ServiceChannel

 

00007ffdb2cf8188       5124        1311744   System.ServiceModel.Security.SecuritySessionServerSettings+SecurityReplySessionChannel

 

Number of Active Sessions

 

 

 

 

To solve this problem, you need to double check the application code of your WCF clients and ensure your WCF clients are closing the created Channels properly after job done. 

 

 

Updated May 04, 2022
Version 2.0
No CommentsBe the first to comment