WCF Based Web App Performance Issue Caused by Hitting MaxConcurrentSessions Limit of WCF
Published May 03 2022 09:31 PM 1,330 Views
Microsoft

A .NET web app hosted on Azure App Service can be built on top of the Windows Communication Foundation (WCF) framework. We've seen a kind of performance/slow response/timeout issue of such web app, which is caused by the MaxConcurrentSessions limit of WCF is hit.

 

The symptom might include that after restarting the web app for a few hours, a percentage of the total requests (e.g. 1/3) get no response from the web server and timeout after 230 seconds. 230 seconds is the Azure default timeout value. The web app's memory usage may be observed to grow continuously since the issue starts. 

 

The default MAX concurrent session setting of WCF server is 5000. If the WCF clients don’t close channels  properly, a lot of sessions which have been completed will not be able to get closed at the WCF server side. This will result in the MaxConcurrentSessions limit of WCF is always hit. Future requests might get 230 seconds timeout results from the WCF server. 

 

We could configure App Service's Auto-Heal rule to collect memory dump for the w3wp instances once the issue happens. You could find Auto-Heal tool from "Diagnose and Solve Problems" on the Azure App Service portal.  Then we can analyze the memory dump in the following way. 

 

  • Download the NetExt extension from here Releases · rodneyviana/netext · GitHub
  • Copy the NetExt.dll file to WinDbg installation folder, e.g. C:\Program Files\Debugging Tools for Windows (x64)\
  • Run WinDbg, open the memory dump file, load the NetExt dll by 0:000> .load NetExt
  • Run 0:000> !windex
  • Run 0:000> !wservice

When issue happens, you may see result like blow 

 

Weizhen_Sun_1-1651638412394.png

You can see the current session count is 5000 and the Max session count is also 5000. 

You can run !dumpheap -type System.ServiceModel.Security.SecuritySessionServerSettings to check Wcf objects. 

Weizhen_Sun_0-1651574392320.png

 

SecuritySessionServerSettings is the important object of the WCF session/connection. You can view the activeSessions object within the SecuritySessionServerSettings object. The "Count" value of the activeSessions object contains the number of the active WCF sessions. It also equals to the number of the System.ServiceModel.Security.SecuritySessionServerSettings+SecurityReplySessionChannel objects. 

 

A comparison of two memory dumps of w3wp when issue happened and not happened.

 

Dump 1# - Working (Issue not replicated)

Time      

Tue Nov  9 08:34:53.000 2021 (UTC + 8:00)

WCF sessions

0:000> !wservice

Sessions/Max   

0n2230/0n5000   

    

Number  and Size of objects related with WCF sessions

MT    Count    TotalSizeClass Name

 

00007ffdb2d18940     2238       304368   System.ServiceModel.Channels.TransactionChannelListener`1+TransactionReplySessionChannel[[System.ServiceModel.Channels.IReplySessionChannel, System.ServiceModel]]

 

00007ffdb2912208     8988       2516640 System.ServiceModel.Channels.ServiceChannel

 

00007ffdb2cf8188      2238       572928   System.ServiceModel.Security.SecuritySessionServerSettings+SecurityReplySessionChannel

 

Number of Active Sessions

 

Weizhen_Sun_0-1651570371593.gif

 

 

 

 

Dump 2# - Issue (replicated)

Time

Tue Nov  9 18:03:55.000 2021 (UTC + 8:00)

WCF sessions

0:000> !wservice

Sessions/Max         

0n5000/0n5000         

Number  and Size of objects related with WCF sessions

MT    Count    TotalSizeClass Name

 

00007ffdb2d18940     5001        680136     System.ServiceModel.Channels.TransactionChannelListener`1+TransactionReplySessionChannel[[System.ServiceModel.Channels.IReplySessionChannel, System.ServiceModel]]

 

00007ffdb2912208     11530      3228400   System.ServiceModel.Channels.ServiceChannel

 

00007ffdb2cf8188       5124        1311744   System.ServiceModel.Security.SecuritySessionServerSettings+SecurityReplySessionChannel

 

Number of Active Sessions

 

Weizhen_Sun_1-1651570531413.gif

 

 

 

To solve this problem, you need to double check the application code of your WCF clients and ensure your WCF clients are closing the created Channels properly after job done. 

 

 

Co-Authors
Version history
Last update:
‎May 03 2022 09:38 PM
Updated by: