Forum Discussion
BrowserMetrics
Hey folks,
Quick update here from the team - they're still investigating but have requested some additional information. Even if you've submitted already, if you don't mind submitting again with the new info we'd really appreciate it!
- On edge://support, do you see a Client Id? What is it?
- The list of PMA files, with time stamps. Mainly the count of files and oldest date would be interesting.
- Run these from a cmd prompt and share the results:
- reg query "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion" /v CurrentBuildNumber
- reg query HKLM\SOFTWARE\Policies\Microsoft\Windows\DataCollection
- reg query HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Microsoft\EdgeUpdate\ClientState /s
- reg query HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Microsoft\EdgeUpdate\ClientStateMedium /s
- edge://local-state/
- edge://version/?show-variations-cmd
If you could head to edge://policy/ and submit feedback (Alt + Shift + I), check to include the screenshot as well as diagnostic data, attach the information requested above, and add the string "TC0317AR-1010LUO-TMEP908" so we can link it to this issue. If you'd like the team to be able to reach out directly, please check to include your email as well.
- JimGrishamApr 01, 2021Iron Contributor
Alexandra-R (and Edge team devs):
Other things of note that I've recently noticed (or heard from others):
Spoiler- Edge seems to clean up these files upon launch. It knows how to do that.
- Edge behaves differently depending on version
- For me, it occurs in:
- Edge Stable: Version 89.0.774.63 (Official build) (64-bit)
- Edge Canary: Version 91.0.837.0 (Official build) (64-bit)
- It does not occur in:
- Edge Beta: Version 90.0.818.22 (Official build) beta (64-bit)
- Therefore, it seems there is either
- a regression between 90.0.918 and 91.0.837, or
- a difference in 'feature flag' or other configuration
- I did notice that the Beta version ran with the following command line argument that was absent from the Stable version
- "--monitor-self"
- For me, it occurs in:
- At first I thought Edge might be losing a 'race condition' with Windows Defender Real Time Protection (RTP); that is, Defender locks the file and Edge can't delete it (see detail [1] below)
- That seems unlikely, though, since the Beta ring works fine with Defender
- Instead, it appears that the Stable version (and possibly the Canary one as well) aren't even _trying_ to delete the old files until after the next time Edge is completely closed and re-opened (... which for some people can be hours to weeks!) (see detail [2] below)
- Perhaps unrelated, there was a mention of this in one of the file access traces - perhaps it is another lead for the devs:
msedge.dll - RelaunchChromeBrowserWithNewCommandLineIfNeeded
To recap this entire discussion, it seems there are two (separate but related) issues that need to be fixed:
- Edge is keeping more than one BrowserMetrics file around, and for far too long
- As I understand it, having more than one file in the BrowserMetrics directory is designed to only ever be a short-term (i.e. 30-60 minutes max) occurrence
- For some combination of builds/users/conditions, Edge is creating these new files at a very high rate (6-300 files / hour).
- Even if the previous issue is corrected, this one is arguably more important and will have more impact on the overall UX of Edge.
- Whether or not they are full of data, that can result in a ton of small disk accesses.
- Pity those unfortunate students using low-end laptops* with spinning HDDs!
- Memory-mapped files like this sometimes get written to very often, since the data is saved in very tiny chunks.
- Based on some of the reports from above, one can imagine the poor little hard drive heads bouncing back-and-forth nearly constantly, impacting system responsiveness and the user's overall opinion of Edge
- An average laptop HDD may only get 60 operations, likely fewer, each second, for the entire operating system and all applications
- The fastest server HDDs still only allow a maximum of about 200 operations / second
- As I have previously mentioned (ok, mildly ranted), each new file can drive activity from filesystem, antivirus/Defender, Windows Auditing, disk indexing, and potentially other processes. To the extent that most of these are reads, their biggest negative impact relates to HDDs.
- * some customers may even have Windows Virtual Desktop instances backed by HDDs, driven by inertia or Azure budgets
- This causes unnecessary wear on consumer SSDs as well.
- The primary concern here is with the frequent small disk writes; read cost / wear is fairly insignificant
- (I believe Chromium on Android turns this feature off altogether, choosing instead to just keep the data in RAM until it needs to be uploaded.)
- Pity those unfortunate students using low-end laptops* with spinning HDDs!
Thank you for all the attention you and your team are giving to this!
Happy hunting,
- Jim
Detail [1] from item (3) in above list:
Spoiler- Edge creates the 4.0 MiB PMA (Persistent Memory Access) file named "BrowserMetrics-spare.pma" in the "%LocalAppData%\Microsoft\Edge\User Data" directory.
- Edge moves that file to the "%LocalAppData%\Microsoft\Edge\User Data\BrowserMetrics" directory and renames it.
- Edge loads that re-named file as a memory-mapped file
- After some time, the 'spare' file from step 1 is re-created
- If Windows Defender is running,
- writes to this file are sometimes / always accessed filtered through the Defender (MsMpEng.exe) process (Synchronous Paging I/O)
- If Windows Defender ATP (Advanced Threat Protection) is running,
- that process (MsSense.exe) requests an 'opportunistic lock (oplock)' [FSCTL_REQUEST_OPLOCK IOCTL (winioctl.h)] while it is scanning
- Before that scan is complete (and therefore the oplock is released), Edge (msedge.exe) closes the file
Detail [2] from item (5) in above list:
SpoilerBeta - Version 90.0.818.22 (Official build) beta (64-bit) --> Normal (working) process, that prevents build-up of "BrowserMetrics-hhhhhhhh-hhhhh.pma" files: CreateFile Desired Access: Generic Read/Write, Disposition: OpenIf, Options: Synchronous IO Non-Alert, Non-Directory File, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: 0, OpenResult: Created CreateFileMapping SyncType: SyncTypeCreateSection, PageProtection: PAGE_EXECUTE_READWRITE Result: FILE LOCKED WITH WRITERS QueryStandardInformationFile AllocationSize: 0, EndOfFile: 0, NumberOfLinks: 1, DeletePending: False, Directory: False SetEndOfFileInformationFile EndOfFile: 4,194,304 CreateFileMapping SyncType: SyncTypeOther ReadFile Offset: 0, Length: 32,768, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Priority: Normal ... CreateFile IRP_MJ_CREATE QueryAttributeTagFile IRP_MJ_QUERY_INFORMATION Type: QueryAttributeTagFile, Attributes: AC, ReparseTag: 0x0 SetDispositionInformationEx IRP_MJ_SET_INFORMATION FILE_DISPOSITION_DELETE, FILE_DISPOSITION_POSIX_SEMANTICS, FILE_DISPOSITION_FORCE_IMAGE_SECTION_CHECK IRP_MJ_CLEANUP CloseFile IRP_MJ_CLOSE
When running the Stable ring version of Edge, "SetDispositionInformationEx" never appears and the files persist.- dave260Apr 02, 2021Brass ContributorThanks Jim for all the updates and spoiler alerts! Good information.
I will quickly add at this point, I am using Edge Beta now. It seems to work well, but I haven't tested every possible issue. Interesting that this is the third day I have used Beta, and there is only ONE file in Browser Metrics. This is the third day I have been using it. On the other hand, Stable Edge has 854 BrowserMetric entries for 3/31-4/1 minimal use.
-- Dave - lexcynApr 01, 2021Iron Contributor
JimGrisham I haven't enabled compression because I am on a corporate device where I do not have that ability - besides, the root cause should be fixed so I will leave as is.
- dave260Apr 01, 2021Brass Contributor
- lexcynMar 30, 2021Iron ContributorThanks - done as requested.
I just cleared out the files a couple of days ago and already up to over 12k new files, all spaced 1 day apart. Added all the details into a txt file in the upload!- JimGrishamMar 31, 2021Iron Contributor
lexcynHow far apart are the dates/times on your files? [Unless my quick math is wrong, 12,000 files over 2 days (48 hours) would be 250/hour, or a new file every 15 seconds or less. Even if over 4 days (96 hours), that is 125 files/hour, still over 2 new files _per minute_.]
dave260 Here's probably why the NTFS file compression isn't working for you:- When NTFS compression is enabled on a file:
- The file is broken up into small sections
- Every time one of those sections is written to,
- The data for that section (including any changes being saved) are compressed in RAM.
- If the compressed size is smaller than the size of the uncompressed data, the compressed section is written to disk.
- If the compressed size is about the same or larger than the uncompressed data, that section is written uncompressed
- The data for that section (including any changes being saved) are compressed in RAM.
- What I think Edge is doing:
- Creates a new empty 4MB file
- 'Memory-maps' that file to an area of memory so that Edge can write to it directly
- Each time it has a drop of 'metrics' data, it stores it in RAM
- Since the file is 'memory-mapped', the system (Windows' virtual memory subsystem and/or the CPU's internal Memory Management Unit) automatically writes the changed data to disk, some small number of bytes at a time
- Since only a tiny bit of data is changing at any one moment, NTFS compression doesn't bother with it (guesses: because it's either too small, because by itself it doesn't compress well, or because it never really works for memory-mapped files)
If you were to turn off compression for your entire BrowserMetrics folder and its contents, and them immediately turned it back on, I think you'd find that all existing files would be greatly compressed, but new files would not. So much for that idea!
- dave260Apr 01, 2021Brass ContributorThanks Jim for the update and details! I will attempt to try your ideas, including the other versions of Edge in your other post above. I like your reasoning about why the auto compress may not compress as expected. Brilliant! I didn't know that. Since I had good results with compression at first, I was surprised when I saw almost no compression today. Other comments above in your later post.....
-- Dave
- When NTFS compression is enabled on a file: