ATP Sensor failed upgrade to 2.198.16173.18440 on Win2012

[RNalivaika]

Hi all, I have a customer running multiple AD Domain Controllers on windows server 2012, 2016 and 2019. ATP sensor version 2.197.16100.44617 was working fine, but a few days ago it started automatic upgrade to 2.198.16173.18440, the new sensor service "Azure Advanced Threat Protection Sensor" cannot start. Application event log also shows a variety of error messages from soure 'Perflib'. This is new, as the 2012 domain controllers were working fine and had no errors in Application log prior to ATP Sensor upgrade. Has anybody experienced the same issue? 

PS1: the new ATP sensor version on windows 2016 and 2019 domain controllers works fine.

PS2: windows 2012 servers running january and february patches.

-Ruslan

[#2]

Uninstalling ATP Sensor and reinstalling results in the same issue. Install wizard says it completed successfully, but the windows service does not start.

[#3]

@RNalivaika  Check the sensor local logs to looks for errors about what is failing it.
https://learn.microsoft.com/en-us/defender-for-identity/troubleshooting-using-logs

If possible paste here the failing call stack and error message.
Another option is to open a support case (might be faster...)

[#4]

Here is the error message from Tri.Sensor log file:
''2023-02-28 08:21:47.7639 Debug EtwListener SetState Creating
2023-02-28 08:21:47.9043 Error EtwTraceDataHelper+NativeMethods System.EntryPointNotFoundException: Unable to find an entry point named 'TdhEnumerateManifestProviderEvents' in DLL 'tdh.dll'.
at TdhStatus Microsoft.Tri.Sensor.EtwTraceDataHelper+NativeMethods.TdhEnumerateManifestProviderEvents(Guid providerGuid, ProviderEventInfo* providerEventInfo, ref int bufferSize)
at IDictionary<EtwEventTypeId, IReadOnlyCollection<EtwEventPropertyInfo>> Microsoft.Tri.Sensor.EtwTraceDataHelper.CreateEtwEventPropertyInfosMappingFromProviderManifest(EtwEventTypeId[] etwEventTypeIds)+(IGrouping<Guid, EtwEventTypeId> groupedEventsByProviderId) => { }
at IEnumerable<TResult> System.Linq.Enumerable.SelectManyIterator<TSource, TResult>(IEnumerable<TSource> source, Func<TSource, IEnumerable<TResult>> selector)+MoveNext()
at Dictionary<TKey, TElement> System.Linq.Enumerable.ToDictionary<TSource, TKey, TElement>(IEnumerable<TSource> source, Func<TSource, TKey> keySelector, Func<TSource, TElement> elementSelector, IEqualityComparer<TKey> comparer)
at Dictionary<TKey, TValue> MoreLinq.MoreEnumerable.ToDictionary<TKey, TValue>(IEnumerable<KeyValuePair<TKey, TValue>> source, IEqualityComparer<TKey> comparer)
at void Microsoft.Tri.Sensor.EtwListener.AddProviderEvents(EtwListenerConfiguration configuration, IEtwEventActivityTranslator etwEventActivityTranslator, IMetricManager metricManager, SensorType sensorType)
at new Microsoft.Tri.Sensor.EtwListener(IConfigurationManager configurationManager, IEtwEventActivityTranslator etwEventActivityTranslator, IMetricManager metricManager, IResearchEnablementEtwEventActivityTranslator researchEnablementEtwEventActivityTranslator, IWorkspaceApplicationSensorApiJsonProxy workspaceApplicationSensorApiJsonProxy)
at object lambda_method(Closure, object[])
at object Autofac.Core.Activators.Reflection.ConstructorParameterBinding.Instantiate()
at void Microsoft.Tri.Infrastructure.ModuleManager.AddModules(Type[] moduleTypes)
at new Microsoft.Tri.Sensor.SensorModuleManager()
at ModuleManager Microsoft.Tri.Sensor.SensorService.CreateModuleManager()
at async Task Microsoft.Tri.Infrastructure.Service.OnStartAsync()
at void Microsoft.Tri.Infrastructure.TaskExtension.Await(Task task)
at void Microsoft.Tri.Infrastructure.Service.OnStart(string[] args)
2023-02-28 08:22:02.4213 Debug ConfigurationManager SetState Creating''

here is the error message from updater log file:
''2023-02-28 08:21:26.4843 Error ServiceControllerExtension ChangeServiceStatus failed to change service status [name=AATPSensor status=Running Exception=System.ServiceProcess.TimeoutException: Time out has expired and the operation has not been completed.
at System.ServiceProcess.ServiceController.WaitForStatus(ServiceControllerStatus desiredStatus, TimeSpan timeout)
at Microsoft.Tri.Infrastructure.ServiceControllerExtension.ChangeServiceStatus(String name, ServiceControllerStatus status, TimeSpan timeout, Nullable`1 awaitedStatus)]''

[#5]

Tricky one. Open a support case for this one. it will need to be escalated to the product group probably.

[#6]

ok, thank you

[#7]

working with ms support is anything other than fast :) sent them logs more than a week ago, still zero progress...

[#8]

Hi @RNalivaika,

What was the solution?

I had similar one some time ago.

Error ServiceControllerExtension Failed to change service status [name=AATPSensor status=Running Exception=System.ServiceProcess.TimeoutException: Time out has expired and the operation has not been completed.

Microsoft.Tri.Sensor-Errors.log file pointed to an error with the WinPcap or NPF driver.

As WinPcap is no longer supported then probably Npcap could be re-installed.

[#9 · Accepted solution]

This issue was escalated via several channels and should have been resolved by now.
Is the sensor still crashing on startup ?

[#10]

@Eli Ofekthe service is starting successfully after update to version 2.199.16251.32043

[#11]

I was already using Npcap, had attempted reinstalling Npcap.
The solution was contacting Microsoft support and after some days they released a new fixed version of the Sensor Service.