Announcing General Availability of FSLogix 2201 (2.9.8111.53415)

vara93 

Another tip for avoiding reboot you can try to use frx.exe stop-agent / start-agent to unlock profile folders.
Btw. about the:

C:\Users\local_username\AppData\Local\Microsoft\Credentials

These are most frequent ones on my side and in 99.9% it was quite easy to release them without reboot.
In most cases there was a lock by some "telemetry" services which are easily restartable...

Btw. when user is logged off correctly, user profile should be unloaded from registry but in your case you still see profile hive in ProfileList and FSLOGIX-Sessions?

- Yes, the records remain. I noticed this seems to happen when a user's session is disconnected by timeout. RDS settings.

Btw. are you using Profile Container only or Office Container or Cloud Cache as well?

- Profile Container only.

What exactly are you redirecting to the local profile?

- https://pastebin.com/tvvabTNX

But even if I turn it off, I see an error with the folders:

C:\Users\local_username\AppData\Local\Microsoft\Credentials
C:\Users\local_username\AppData\Roaming\Microsoft\Credentials

Edit: Did you try "&'C:\Program Files\FSLogix\Apps\frx.exe' list-redirects" command if you can see active redirect there for local_profile folder?

- I did a reboot over the weekend to clear the folders, I'll check back later.

I think these folders:

C:\Users\local_username\AppData\Local\Microsoft\Credentials
C:\Users\local_username\AppData\Roaming\Microsoft\Credentials

not related to Microsoft Office or google chrome browser

The FSLogix developers say that folder freezes are not related to FSlogix.
But why can't they make Microsoft components compatible with their program?

We don't use redirections so folders are almost empty, only some caches or temp files.
Btw. when user is logged off correctly, user profile should be unloaded from registry but in your case you still see profile hive in ProfileList and FSLOGIX-Sessions? Weird as they should be unloaded during logoff.
Maybe FSLOGIX log [C:\ProgramData\FSLogix\Logs\Profile\] shows something relevant?
Btw. are you using Profile Container only or Office Container or Cloud Cache as well?
What exactly are you redirecting to the local profile? Maybe some item there is the culprit why folder is not correctly unmounted... If you have problems with undeleted local_profile every time, I should recommend to create special GPO for specific test user where you will disable redirections (deny permissions for specific user on such redirectionsXML share should be enough to not apply them whithout changing any GPO) and test it if local_profile is correctly cleared. If yes, direct your sight on redirections XML file...

Edit: Did you try "&'C:\Program Files\FSLogix\Apps\frx.exe' list-redirects" command if you can see active redirect there for local_profile folder?

ExSportCZ, hi.

"when you run "mountvol" command, do you see if profile is mounted?"
- No.

\\?\Volume{bb3aa5e4-0000-0000-0000-602200000000}\
C:\

\\?\Volume{934d4f6b-9ba1-11ec-b596-806e6f6e6963}\
A:\

\\?\Volume{934d4f6a-9ba1-11ec-b596-806e6f6e6963}\
D:\

"Please check registry "HKEY_LOCAL_MACHINE\SOFTWARE\FSLogix\Profiles\Sessions" and "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\ProfileList" as well if profile of the affected user is unloaded or not."

https://ibb.co/j4qDC3q
https://ibb.co/PQwvMCF

I have to delete 20 gigabytes every week ~

vara93 

Unfortunately it is normal these handles can't be easily closed without the server reboot.
Hard to guess remotely what's exactly locking it, if some other troubleshooting may help or only reboot is the solution.
Right now in test env I can't simulate it as we currently test private version which don't let me to break profile easily so some fragments will not unload correctly.

Btw. normally you should see more than "Type: File" only (you must use -a parameter), e.g. Mutant/Event/Token. Did you use "-a" param? If yes, then it seems only reboot will help, unfortunately.

Edit: Ok I forced a lock again but another one than your so this one was easily fixable without reboot by restarting a single service:

As you can see almost unrelated service/process released other locks as well, including PID4 ones. If your session host where you have problems is not productive, you can try to stop/restart each non-critical OS processes/services (Antivirus, FSLOGIX,...) and then system services as well (which are restartable, non-critical) and monitor if any of them will release some of the locks for you to find out which process/service is the culprit.
Good luck to find it out to avoid whole server reboot but to be honest I expect you will be forced to reboot it to resolve the "issue".

Wow, I'm making progress. My profile is busy with services:

Netlogon
SamSs

ExSportCZBut after restarting the Netlogon service, it doesn't help to free the profile, it's still occupied by that service. SamSs service I can't restart.

What to do ? UPD:

Until: https://ibb.co/Px9vpjf

After: https://ibb.co/FgvhVW0

I don't see any more services, but pid 4 won't let the folders go.

UPD: https://ibb.co/qp4CGDx

Running the command line from the system (psexec -i -s cmd.exe) did not work.

vara93 

There are locks by SYSTEM only what is not easily possible to unlock.
You must do it via another system services but you need to find them out.
Run "handle64.exe -u -a A.Korotkov"
You need to search for "username" not the "folder". Hopefully you will find something else than PID 4 only. My "redacted" example:

lsass.exe pid: 672 type: Token NT AUTHORITY\SYSTEM 173C: DOMAIN-NAME\affected_user:b5ea29b
svchost.exe pid: 2164 type: Token NT AUTHORITY\SYSTEM 430: DOMAIN-NAME\affected_user:b5ea29b
svchost.exe pid: 7212 type: Token NT AUTHORITY\SYSTEM 54C: DOMAIN-NAME\affected_user:b5ea29b
svchost.exe pid: 8732 type: Token NT AUTHORITY\SYSTEM 754: DOMAIN-NAME\affected_user:b5ea29b
svchost.exe pid: 8732 type: Token NT AUTHORITY\SYSTEM 79C: DOMAIN-NAME\affected_user:5b350b
svchost.exe pid: 8732 type: Token NT AUTHORITY\SYSTEM 7A0: DOMAIN-NAME\affected_user:5b350b
svchost.exe pid: 8732 type: Token NT AUTHORITY\SYSTEM 83C: DOMAIN-NAME\affected_user:5b350b
svchost.exe pid: 8732 type: Token NT AUTHORITY\SYSTEM 868: DOMAIN-NAME\affected_user:b5ea29b
svchost.exe pid: 6492 type: Token NT AUTHORITY\SYSTEM 36C: DOMAIN-NAME\affected_user:b5ea29b
svchost.exe pid: 9480 type: Token NT AUTHORITY\SYSTEM 4AC: DOMAIN-NAME\affected_user:b5ea29b

If you will have some PIDs<>4, find services for them and restart them one after one but after each restart check if locks report changes as not always it is needed to restart all of them as one service may release more locks, including PID 4 ones. Go from bottom to the top in the "handle" list.

Very stupid powershell script which will list services with PIDs found before:

@(672,2164,7212,8732,6492,9480)|%{Get-CimInstance -Class Win32_Service -Filter "ProcessId=$_"}|ft DisplayName,Name,ProcessId,State,AcceptStop,StartName -a

Replace the array of PIDs with your ones.

Hope it helps

vara93 

It is a part of "famous" SYSINTERNALS tools from Mark Russinovich.
https://docs.microsoft.com/en-us/sysinternals/downloads/handle
If I remember right, these were e.g.:

Microsoft Account Sign-In Assistant [wlidsvc]
AppReadiness
BITS
Program Compatibility Assistant Service [PcaSvc]
UserManager

Restarting one after another cleared all the locks, including SYSTEM [Pid 4] ones.

I noticed that sometimes drives with user profiles are not unpinned when logging off.

Result: the session is logged off and the user profile disk is still connected.

Has anyone noticed a similar problem?

Thor92 
If you read my description what is happening in the background you should imagine why such behavior.
The root issue is FSLOGIX don't handle user sessions/processes impersonated from other user sessions (or anytime more sessions with different ID is opened for same user).
If there isn't existing user profile loaded already (handled by FSLOGIX), new background process is loaded with "default network profile". Now when "real/full RDP/RemoteApp" session is loaded, new session is created with different sessionID (handled by FSLOGIX so previous profile is unloaded). Such behavior will broke whole profile functionality that everything will broke (in the new session explorer process crash endlessly, etc...).

In some cases, when it happens, it can be fixed by session host restart but not always it helps as sometimes some fragments will stay in config on session host or in profile so restart will not help either.
Other times it is enough to delete profile so it is recreated but if session host is "broken", user is able to login to the other host but not to the one where such issue with double sessions occurred (issues with import/export registry, etc...).
The good news is MS fixed these broken states which can happen to session hosts or profiles.
Right now fix is in internal/private version only so hopefully will be released publicly soon.
Still there are some other bugs not fixed but great it stopped corrupting session hosts or profiles itself 🙂

We have been good for a couple of weeks.

Turns out that even if you are not actively using Windows Defender on an Azure VM, it is still "Active" to some extent.

Use below to add exclusions to Windows defender (even if you are running another AV):

# Defender Exclusions for FSLogix
$Cloudcache = $false # Set for true if using cloud cache
$StorageAcct = "Storage Account Name" # Storage Account Name
$Share = "File Share"
 
$filelist = `
"%ProgramFiles%\FSLogix\Apps\frxdrv.sys", `
"%ProgramFiles%\FSLogix\Apps\frxdrvvt.sys", `
"%ProgramFiles%\FSLogix\Apps\frxccd.sys", `
"%TEMP%\*.VHD", `
"%TEMP%\*.VHDX", `
"%Windir%\TEMP\*.VHD", `
"%Windir%\TEMP\*.VHDX", `
"\\$Storageacct.file.core.windows.net\$Share\*\*.VHD", `
"\\$Storageacct.file.core.windows.net\$Share\*\*.VHDX"
 
$processlist = `
"%ProgramFiles%\FSLogix\Apps\frxccd.exe", `
"%ProgramFiles%\FSLogix\Apps\frxccds.exe", `
"%ProgramFiles%\FSLogix\Apps\frxsvc.exe"
 
Foreach($item in $filelist){
Add-MpPreference -ExclusionPath $item}
Foreach($item in $processlist){
Add-MpPreference -ExclusionProcess $item}
 
If ($Cloudcache){
Add-MpPreference -ExclusionPath "%ProgramData%\FSLogix\Cache\*.VHD"
Add-MpPreference -ExclusionPath "%ProgramData%\FSLogix\Cache\*.VHDX"
Add-MpPreference -ExclusionPath "%ProgramData%\FSLogix\Proxy\*.VHD"
Add-MpPreference -ExclusionPath "%ProgramData%\FSLogix\Proxy\*.VHDX"}

This will stop Defender interacting with VHD etc. 

Also, look up autoendtasks registry key and apply to your VMs/images - helps with making sure nothing hangs up the profile unload process on user logoff.

After implementing the above we have been okay for a little while with the single user who was affected. 

Too early to call it resolved but so far so good. 

Sounds similar to the issues we’ve been seeing for the last few months. I put In a support ticket about 3 weeks ago to Microsoft and haven’t heard a thing back about it.  Mostly ours is the local profile left but we have had instances of the username folder left and that stops the user from logging in to that machine again due to profile unable to load message at logon.  Like you said the only solution appears to be a machine reboot.  We are still performing nightly checks across 16 machines and rebooting any with leftover profiles.  We are also supposed to be building another 70 machines in another region and that work is on hold as performing checks on 16 servers is bad enough for our ops guys but asking them to check another 70 is not a valid solution