Forum Discussion
Sensor service keeps restarting (after auto upgrade)
Hi all,
I've installed multiple Azure ATP Sensor Setup yesterday on Windows 2019 and 2022 servers. But one is failing to report in the console today.
I've checked the system and the AATPSensor service is always in the starting / stopped / starting state.
The Tri.Sensor-Errors.log shows this:
2024-02-08 13:35:20.1835 Error RegistryKey System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
at object Microsoft.Win32.RegistryKey.InternalGetValue(string name, object defaultValue, bool doNotExpand, bool checkSecurity)
at object Microsoft.Win32.RegistryKey.GetValue(string name)
at byte[] System.Diagnostics.PerformanceMonitor.GetData(string item)
at byte[] System.Diagnostics.PerformanceCounterLib.GetPerformanceData(string item)
at Hashtable System.Diagnostics.PerformanceCounterLib.get_CategoryTable()
at CategorySample System.Diagnostics.PerformanceCounterLib.GetCategorySample(string category)
at CategorySample System.Diagnostics.PerformanceCounterLib.GetCategorySample(string machine, string category)
at string[] System.Diagnostics.PerformanceCounterCategory.GetCounterInstances(string categoryName, string machineName)
at new Microsoft.Tri.Infrastructure.MetricManager(IConfigurationManager configurationManager)
at object lambda_method(Closure, object[])
at object Autofac.Core.Activators.Reflection.ConstructorParameterBinding.Instantiate()
at void Microsoft.Tri.Infrastructure.ModuleManager.AddModules(Type[] moduleTypes)
at new Microsoft.Tri.Sensor.SensorModuleManager()
at ModuleManager Microsoft.Tri.Sensor.SensorService.CreateModuleManager()
at async Task Microsoft.Tri.Infrastructure.Service.OnStartAsync()
at void Microsoft.Tri.Infrastructure.TaskExtension.Await(Task task)
at void Microsoft.Tri.Infrastructure.Service.OnStart(string[] args)
2024-02-08 13:35:29.0122 Error RegistryKey System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
at object Microsoft.Win32.RegistryKey.InternalGetValue(string name, object defaultValue, bool doNotExpand, bool checkSecurity)
at object Microsoft.Win32.RegistryKey.GetValue(string name)
at byte[] System.Diagnostics.PerformanceMonitor.GetData(string item)
at byte[] System.Diagnostics.PerformanceCounterLib.GetPerformanceData(string item)
at Hashtable System.Diagnostics.PerformanceCounterLib.get_CategoryTable()
at CategorySample System.Diagnostics.PerformanceCounterLib.GetCategorySample(string category)
at CategorySample System.Diagnostics.PerformanceCounterLib.GetCategorySample(string machine, string category)
at string[] System.Diagnostics.PerformanceCounterCategory.GetCounterInstances(string categoryName, string machineName)
at new Microsoft.Tri.Infrastructure.MetricManager(IConfigurationManager configurationManager)
at object lambda_method(Closure, object[])
at object Autofac.Core.Activators.Reflection.ConstructorParameterBinding.Instantiate()
at void Microsoft.Tri.Infrastructure.ModuleManager.AddModules(Type[] moduleTypes)
at new Microsoft.Tri.Sensor.SensorModuleManager()
at ModuleManager Microsoft.Tri.Sensor.SensorService.CreateModuleManager()
at async Task Microsoft.Tri.Infrastructure.Service.OnStartAsync()
at void Microsoft.Tri.Infrastructure.TaskExtension.Await(Task task)
at void Microsoft.Tri.Infrastructure.Service.OnStart(string[] args)
2024-02-08 13:35:37.9346 Error RegistryKey System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.I've tried rebooting the server, but that didn't fix the problem.
Then I removed the installation, and reinstalled the sensor.
That didn't help, either.
Looks like there was an update installed after the initial setup yesterday, since there were two folders in C:\Program Files\Azure Advanced Threat Protection Sensor :
2.227.17547.62185
2.228.17612.22841
I also tried to solve the problem with a re-downloaded installer package from today (was a different size) but that didn't help. The version installed is the second one from above now.
Any hints on the error message?
Thanks in advance
Chris
- EliOfek
Microsoft
ChrisVie
Please run from a powershell session on the failing machine this command:(New-Object System.Diagnostics.PerformanceCounterCategory("Network Interface")).GetInstanceNames()
And let me know the result.
I can't see any relation to version 2.228, it looks like we are failing when trying to look up instances
of the "Network Interface" perf counter category.
I am guessing something is wrong with this category registration, that fails the interface we are using.
The powershell command above does (almost) the same thing, so if there is an OS issue it should manifest there as well...Thanks for this Query! This led me to the solution. I had a similiar case. OS 2019. 1 DC working fine, other DC gave problems with ATP sensor. Same logfile errors as the OP. You pointed me in the right direction: Sensor wouldn't start, since there is something wrong on the OS level with a performance counter. Open up Performance Monitor and you'll get an error about the Network Interface not being able to get data.
Solution:
-lodctr /E:TCPIP
This uses the lodctr command, with the Enablement option, for the Performanc Counter called TCPIP. This is ultimately the Network Interface performance counter since it uses the %SystemRoot%\System32\perfnet.dll file. If you run 'Lodctr /Q' is shows the list of available performance counters, as well as their state (enabled/Disabled). The Tcpip/network interface counter was disabled.
EliOfek : i don't read in the MDI/Azure Advanced Threat Protection Sensor documentation, that this is a requirement. Is it possible to update this? Or create an extra check during installation for the Enablement of this performance counter?
- EliOfekThis should be enabled by default. the sensor is relying on many things in the OS which are enabled by default and the common case it stays like that.
When this counter is disabled it's usually due to an error.
Anyway, I have forwarded this feedback to consider it for the docs.
- EliOfekIt seems to work from powershell.
So either its related to running as local system or specific to dotnet interface.
Did you try to reboot the machine ?
- Did you check if there are updates?
