FSLogix Profile Hangs - "Reset to device, \Device\RaidPort1, was issued."

Copper Contributor

 

Google scouring has been done. This is my last ditch effort before trying to open a ticket with Microsoft (not fun).

 

Issue:
Randomly and without any cause I can find, a mounted VHD FSLogix profile will hang and not disconnect cleanly from a Citrix VDA server.

 

The only indication I have that this is happening, before the server craters, is the following in the SYSTEM event log:

 

"Reset to device, \Device\RaidPort1, was issued."
Event ID: 129
Source: vhdmp

 

This shows up about every 30 seconds.

 

This sets off a deterioration of the OS itself.

 

If caught early, I can RDP to the server.

 

If not, RDP fails and sits at a black screen trying to connect.

 

The only way to get server back to normal is a hard reboot.

 

Users on the server in a Citrx session do not seem to notice an issue.

 

If I can RDP to the server, I can identify the profile having the issue. It will show session disconnected (most of the time), however the session cannot be forced to log off. Trying to kill processes used by the profile will work until it doesn't. Several will say "access denied".

 

Eventually the server deteriorates to stopping updating the Citrix Director/Manager and reporting accurate data on who is logged on, etc.

 

Also of note, under Disk Management, you can see mounted VHDs normally. When the server is in this state, Disk Management hangs trying to open.

 

Under System\Advanced System Settings\User Profile --> selecting Settings, which usually takes you to profiles on the system, also hangs.

 

It's as if the entire system that manages profiles and mounted VHDs is choking.

 

I have about a dozen Citrix VDAs and this happens on all of them randomly at some point. Averaging 2-3 times a week.

 

All are Windows Server 2012 R2 <-- Yes, I know. It is on this version for "reasons".
VDAs are VMware VMs with plenty of resources.
Citrix version 1912.0.2000.2345
FSLogix version 2.9.7838.44236 (latest version as of now).

 

\Device\RaidPort1 is an alias for all of the mounted VHD profiles, as I have found doing a deep dig. So I'm very confidence that this is a profile problem, FSLogix, or some combination of something with the OS, an app or Citrix (see how I narrowed that down?). The profiles that get stuck are also random. No common users I can find causing it.

 

Looking for another clue....

4 Replies

@jeffstamp 

I feel your pain. Same issue here.

 

I removed FSLOGIX and put Citrix UPM in hopes of a permanent solution but it still occurs to me.

 

windows 2016 / Citrix VDA 1912 LTSR CU2, published apps only.(no desktop)

 

i still have no clue of why it does happen, i suspect the "sharing" of the profiles (same user , same profile on 2 differents citrix server)

Did you ever find a Solution for this? We experience the exact same behavior on Windows Server 2022 with Citrix Virtual Apps LTSR 2203 CU4 and FSLogix 2210 hotfix 2 (2.9.8612.60056).

@Andreas_Zuegler 

 

Through the FSLogix team via Microsoft Support, we were given a workaround that worked.

 

That is to turn off Hiding Rules in FSLogix completely. Essentially not using that feature at all. Since then, that error has stopped.

 

But we had to go remove a bunch of Start Menu shortcuts we didn't want users trying or were useless to them. Definitely a pain, but it stopped the bleeding.

 

I hope this helps.

@jeffstamp 

 

Thanks for your answer. In our case it was a totally other issue. I want to share it, so it maybe helps someone who experience the same issue.

 

We found out that the fileserver, where the share for the fslogix VHDX files are stored, had memory usage from over 95% from time to time. At the exact same timetamp we saw warnings from NTFS (Event ID 140) in the Eventlog of the fileserver and the "Reset to device, \Device\RaidPort1, was issued." Event log entry on the affected Terminalserver. It seems that fslogix somehow could not finish the VHDX mount or unmount process successfully at the time when not enough memory was available on the fileserver and after that it started to act awkwardly on the affected terminalserver resulting in the exact behavior you described. Adding more memory to the fileserver completely solved our problem.

 

BR,

Andreas