SOLVED

Windows Server 2012 R2 - memory leak SysTrace.sys

Copper Contributor

Hello,

since last month our server started having issue with memory leak all of sudden where non-paged pool increases in size to about 25GB filling up the whole memory, making the system unresponsive.

I traced it with pooltag, the culprit is in tag "FDRo", searching through sys files it pointed to "SysTrace.sys", which is supposed to be "Microsoft MSN Flight Data Recorder". The server runs normally for about 3-4 days, then within couple of hours the memory gets filled up and server crashes.

Is there any way to disable the Flight Data Recorder or fix the driver? Also it is a VM machine running on Hyper-V

 

Thank you

 

mstsc_2023-03-30_09-33-49.png

 

mstsc_2023-03-26_16-33-56.png

 

mstsc_2023-03-21_18-42-02.png

 

11 Replies
best response confirmed by JardaTesar (Copper Contributor)

@Dave Patrick How could I miss this post. Thank you very much. I uninstalled the Software Certification Toolkit and renamed the sys file. Hopefully it will fix the issue.

No worries, glad to hear, you're welcome.  

 

 

Hi, so unfortunately it didn't help. I uninstalled the Certification Toolkit, deleted the Systrace.sys, rebooted the server, it was ok until today, but now the non-paged started increasing again, still with the same pool tag "FDRo". I wasn't able to find any other sys file.

The only other thing to try may be to stand up a new one without the Certification Toolkit for testing.  

 

 

@Dave Patrick thanks for the response. What do you mean by stand up a new one? Do you mean to set up a new VM? On the current VM I removed all traces of Certification toolkit (don't need it anymore) and removes the systrace.sys, however the problem still keeps coming back, with the same pool tag, but I can't find any other sys with this pool tag.


What do you mean by stand up a new one? Do you mean to set up a new VM? 

Yes, I did. 

 

 

There are other VMs running on the same hypervisor and they are running fine. To set up a new VM with same services would be very hard to do, because it is running SAP server with MSSQL, so to configure it all out would be a pain. I will do some more debugging to see if I can find the culprit. What bugs me is that there is the FDRo tag and I can't trace where it is coming from. Thanks for you help anyway.

what does the

findstr /m /l FDRo *.sys

return with now? has it been rebooted?

 

 

Yes it has been rebooted. And this doesn't find anything now. I used this command previously to find the SysTrace.sys. I even tried to run this systemwide with the `/s` arg, but it just ran for couple of hours and didn't produce any output. Strange thing is that the system is fine for 3-4 days, non-paged pool hovering around 500MB, then all of sudden within couple of hours it starts raising and crashes.
1 best response

Accepted Solutions
best response confirmed by JardaTesar (Copper Contributor)