Forum Discussion
What causes SMB server error STATUS_INSUFFICIENT_RESOURCES (0xc000009a) ?
I’m looking for advice on understanding this SMB error: STATUS_INSUFFICIENT_RESOURCES (0xc000009a). I have also posted this to https://www.reddit.com/r/WindowsServer/comments/16zc6pa/windows_file_server_what_causes_smb_error_status/ , but posting here after not receiving any useful responses for a week.
We have a Windows Server 2019 web server (SMB client) running Tomcat software which, among other things, has thousands of SMB folders open from our Win 2019 file server to monitor for changes.
Approximately every 2 weeks, the web server starts getting errors opening new files/folders from that file server. Wireshark reveals that the file server responds to our web server with: NT Status: STATUS_INSUFFICIENT_RESOURCES (0xc000009a). This same error happens to all users from that particular web server machine, but those same users can access the same folders from any other machine. This might sound like a problem with that web server, but we have tried restarting the web server and this has no effect with same errors continuing after the reboot, however, the issue is always immediately resolved after restarting the file server.
I am not sure which resources it is reporting is insufficient: it’s certainly not RAM, CPU or disk space, of which there is plenty.
Windows File Explorer on our web server (SMB client) shows different error messages including:
Network Error: Windows cannot access xxx Check the spelling of the name. Otherwise, there might be a problem with your network. To try to identify and resolve network problems, click Diagnose. Error code 0x80070035”
Open Folder: xxx is not accessible. You might not have permission to use this network resource. Contact the administrator of this server to find out if you have access permissions. The remote procedure call failed and did not execute.
In either case, the underlying SMB protocol error messages captured by Wireshark is the same “STATUS_INSUFFICIENT_RESOURCES (0xc000009a)” message.
On the web server SMB client, no entries are logged to the Event Viewer log under Applications and Services Logs / Microsoft / Windows / SMBClient when the unsuccessful connection attempts are made. On the file server, no entries are logged to the SMBServer logs either.
The incident has occurred many times: approx every 2 weeks, e.g. 4 Sept, 20 Sept, 3 Oct.
During the 20 Sept incident, the file server showed the below in MMC with “Shared folder” snap-in added:
The user the Tomcat web server process is running at is the one with the 8562 open filesThis is the "Shares" section of the "Shared Folders" snap-in.
Most open files were in fact folders with 0 locks and open in Read Mode as expected with the web server monitoring those folder paths for changes.
Issue occurred again 3 October, this time the following diagnostic info was captured on the web server. Again the username the Tomcat web server runs under is the one that had the 9223 NumOpens.
PS C:\Windows\system32> Get-SmbConnection ServerName ShareName UserName Credential Dialect NumOpens ---------- --------- -------- ---------- ------- --------
xxxyyy-fss-001 soeapps$ xxx\UsernameA xxx.LOCAL\UsernameA 3.1.1 1
xxxyyy-FSS-003 IPC$ xxx\UsernameA xxx.LOCAL\UsernameA 3.1.1 1
xxxyyy-FSS-003 TMSImages$ xxx\usernamessssdds.eng xxx.LOCAL\usernamessssdds.eng 3.1.1 9223
PS C:\Windows\system32> Get-SmbClientConfiguration
ConnectionCountPerRssNetworkInterface : 4
DirectoryCacheEntriesMax : 16
DirectoryCacheEntrySizeMax : 65536
DirectoryCacheLifetime : 10
DormantFileLimit : 1023
EnableBandwidthThrottling : True
EnableByteRangeLockingOnReadOnlyFiles : True
EnableInsecureGuestLogons : False
EnableLargeMtu : True
EnableLoadBalanceScaleOut : True
EnableMultiChannel : True
EnableSecuritySignature : True
ExtendedSessionTimeout : 1000
FileInfoCacheEntriesMax : 64
FileInfoCacheLifetime : 10
FileNotFoundCacheEntriesMax : 128
FileNotFoundCacheLifetime : 5
KeepConn : 600
MaxCmds : 50
MaximumConnectionCountPerServer : 32
OplocksDisabled : False
RequireSecuritySignature : False
SessionTimeout : 60
UseOpportunisticLocking : True
WindowSizeThreshold : 1
PS C:\Windows\system32> Get-SmbClientNetworkInterface
Interface Index RSS Capable RDMA Capable Speed IpAddresses Friendly Name
--------------- ----------- ------------ ----- ----------- -------------
10 True False 10 Gbps {aaaa::bbbb:cccc:dddd:eee, 10.10.1.155} Ethernet0 On the file server, the following diagnostic info was captured:
PS C:\WINDOWS\system32> Get-SmbServerConfiguration AnnounceComment : AnnounceServer : False AsynchronousCredits : 512 AuditSmb1Access : False AutoDisconnectTimeout : 15 AutoShareServer : True AutoShareWorkstation : True CachedOpenLimit : 0 DurableHandleV2TimeoutInSeconds : 180 EnableAuthenticateUserSharing : False EnableDownlevelTimewarp : False EnableForcedLogoff : True EnableLeasing : True EnableMultiChannel : True EnableOplocks : True EnableSecuritySignature : False EnableSMB1Protocol : True EnableSMB2Protocol : True EnableStrictNameChecking : True EncryptData : False IrpStackSize : 15 KeepAliveTime : 2 MaxChannelPerSession : 32 MaxMpxCount : 50 MaxSessionPerConnection : 16384 MaxThreadsPerQueue : 20 MaxWorkItems : 1 NullSessionPipes : NullSessionShares : OplockBreakWait : 35 PendingClientTimeoutInSeconds : 120 RejectUnencryptedAccess : True RequireSecuritySignature : False ServerHidden : True Smb2CreditsMax : 8192 Smb2CreditsMin : 512 SmbServerNameHardeningLevel : 0 TreatHostAsStableStorage : False ValidateAliasNotCircular : True ValidateShareScope : True ValidateShareScopeNotAliased : True ValidateTargetName : True
What else can I do to troubleshoot? Any other diagnostic info I should collect next time the incident occurs? Is there some kind of limit I need to increase considering there is plenty of system resources available. Any ideas where to go from here?
2 Replies
Hi DanDan320,
this can be caused by a lot of things, like too many open folders, too many connections, or not enough CPU, memory, or disk space...
Here are a few more things you can do to troubleshoot the problem:
- Check your server resources. Make sure your server isn't overloaded with CPU, memory, or disk usage. You can use tools like Windows Performance Monitor to check this.
- Check your SMB connections. See how many connections your server is handling at once. You can use the netstat command to check this. If you're seeing a lot of connections, it could be maxing out your server's resources.
- Check your server settings. There might be limits on how many connections or open files your server can handle. If these limits are too low, you may need to raise them. You can usually find these settings in the server's management console.
- Check your network. Make sure there are no problems with your network, like slow connections or lost data. This can mess up your file sharing. You can use tools like Ping and Tracert to test your network connection.
- Monitor your resources. Use tools to keep an eye on how your server uses resources over time. Watch for any big spikes before the problem happens again. This can help you identify the source of the problem.
- Review your file sharing settings. Check things like how long sessions last and how files are locked. You may need to adjust these settings to reduce the load on your server.
Please click Mark as Best Response & Like if my post helped you to solve your issue.
This will help others to find the correct solution easily. It also closes the item.If the post was useful in other ways, please consider giving it Like.
Kindest regards,
Leon Pavesic
(LinkedIn)- Check your server resources - as already mentioned in original post, issue is not CPU, RAM or disk. Of course I checked Task Manager, Resource manager, monitoring server, etc. as the first step.
- Check your SMB connections - netstat shows about 20 active connections including just 1x from the problematic machine. Get-SmbSession lists about 10 sessions, most with just a few files open, but the problematic client has 9011 files open in read mode, due to monitoring all those paths for changes, but as I said before, plenty CPU, RAM, disk, so I don't see this as a problem, unless there's some configured limit.
- Check your server settings - This very obvious advice doesn't help without specifics. I did post the output of Get-SmbServerConfiguration above. Is there some other server settings I should check?
- Check your network. - My 10GbE connection is perfectly fine, no packet drops, etc.
- Monitor your resources. - again fine.
- Review your file sharing settings. - yeah, again, any specifics? Where, which command or tool how?
