AppFabric Caching Service crashes (Event ID: 111)

AppFabric helps IT professionals to build scalable web applications on IIS. It consists of hosting and caching features. In this post, I will mention some possible solutions to AppFabric Caching Service crashes.

AppFabric Caching architecture (Source)

If the caching has issues, check the Event Viewer logs in the containers below:

• Windows Logs > Application
• Windows Logs > System​
• Applications and Services Logs > Microsoft > Application Server-System Services​

In the Event Viewer logs, it is likely that you will see one or more of these error messages:

Event ID 111:
AppFabricCachingService.Crash Microsoft.ApplicationServer.Caching.DataCacheException: ErrorCode:SubStatus:Service initialization failed. No user action required.

Event ID 6:
30ecac98000000000000000000000000
Microsoft.Fabric.Common.OperationCompletedException: Operation completed with an exception —> System.TimeoutException: The operation has timed out.

The root cause and the solution

Some findings and fixes are below. Each of them can be implemented individually.

Service account

Make sure the AppFabric Caching service uses a managed account (Instructions).

Best practice for the Caching service account:

A single account should be used for all Service Applications, named Service Application Pool account. This allows the administrator to use a single IIS Application Pool for all Service Applications. In addition, this account should run the following Windows Services: SharePoint Search Host Controller, SharePoint Server Search, and Distributed Cache (AppFabric Caching Service).

Source

Account permissions

NT Authority\System on the SQL AppFabric Database should have Data Reader / Data Writer permissions.

Also, in PowerShell, execute Grant-CacheAllowedClientAccount command for NT Authority\System account.

Other possible fixes

• Check if all cluster nodes are online
• Deactivate the antivirus temporarily
• Keep the operating system up-to-date
• Check if the limit for number of servers allowed in the farm is not exceeded
• Deactivate Windows Firewall temporarily. If there is a non-Windows firewall, try opening these protocols/ports: SMB, SMB2 and ICMP protocols TCP ports (22233, 22234, 22235 and 22236) Or 445 and the ICMP protocol TCP ports (22233, 22234, 22235 and 22236)