I'd like Windows developers to look at the issue reported here.
I reported this twice already:
https://answers.microsoft.com/en-us/windows/forum/windows_10-performance/windows-is-leaking-memory-w...
https://learn.microsoft.com/en-us/answers/questions/1601862/windows-is-leaking-memory-when-reading-r...
And originally posted at https://internals.rust-lang.org/t/high-memory-usage-on-random-file-reads-using-seek-read-windows/203...
Support agent in live chat recommended to post it here instead, so here I am, posting a third time with hopes to reach someone who can fix this issue.
Tl;DR: A small Windows app (in the internals.rust-lang.org thread linked above) reads tiny chunks of a large 500G file at random and uses 0.5M of RAM, while total memory used by Windows increases by ~1G instead. This is MUCH worse when even larger files are involved, similar app easily makes Windows use over 100G of RAM for no good reason. For some users this means active swapping to disk and sluggish experience, some just run out of memory and crash. Reproducible on Windows 10 and 11.
Things I tried and learned:
- Happens both with and without FILE_FLAG_RANDOM_ACCESS, but intention is to use FILE_FLAG_RANDOM_ACCESS because reads are in fact random and there is no point in caching anything
- Memory is used by memory mapping in the kernel (not process) according to RamMap and VMMap that I did NOT request on purpose
- I have seen memory issues on Windows due to how poorly it handles memory-mapped I/O and moved away from it, but here we are again
- FILE_FLAG_NO_BUFFERING is not a good solution because due to its alignment requirements requires my cross-platform app (that works perfectly fine on Linux and macOS) to be substantionally refactored due to Windows bugs that should not exist in the first place
P.S. I picked "Community" label as the least irrelevant, support representative said "Just post it." 🤷, so please don't just delete this.
