Active OneDrive makes whole environment 'sluggish'

[EBRIN2233]

Our client is using AVD for about 5 weeks in 'production' now. It is a smaller company with about 40 users. We have 6 VM's in AVD (D4s_v5) in use at the moment. We have tried D8s_v5 without any improvement.  
But we see memory and cpu usage at high levels most of the time with OneDrive at the top of the processes. Every server feels sluggish at the moment. Starting Excel takes about 30-40 seconds, Word about the same time. Opening a Word document takes about 20-25 seconds to open and opening folders from the right pane (yes only the right pane) in explorer takes abount 7-15 seconds per folder.

All those folders are folders from SharePoint document libraries. 
It doens't matter if a folder is fully synced with the server or it isn't. 

We are using FSlogic with profiles stored on azure. 

I was reading a blog of Christian Brinkhoff about almost the same kind of issues but can't really pinpoint the exact cause or find a solution for this issue.

Can you help me out?

[#2]

@EBRIN2233

I thought this is used for file sync, what kind of resource in high from your environment?

[#3]

This could be several things. Firstly, if you take OneDrive out of the equation, is still slow?? I have never seen OneDrive slow things down.

Few things to check:
1. How many users are you getting per host? With a D4S_5 try it with a 1:1 ratio to set it to 4 users per host and see if you still experience issues.
2. Have you implemented the FSLogix exclusions? I would strongly suggest doing so if not.
3. What storage are you using for the VMs, you should be at least Standard SSD
4. Try the D4DS_V5 SKU as that has a temp disk so your pagefile disk will have much more IOPS so hopefully better experience
5. If you open a local Word or Excel document is it still slow?
6. What storage is your FSLogix profiles held on? Should be at least Premium SSD.
7. If you exclude FSLogix from the equation, is it still slow?? You can do this by adding users to the local FSLogix exclude group on the session hosts.

This should help pinpoint where the issue is 🙂

[#4]

@virtualmanc

Disabeling OneDrive makes the servers 'fly'. Enabled slows them down incredibly.
It doesn't matter if 1 or 5 users are on the server. We upgraded to D8s_V5 without improvement.

Storage for servers and FSLogic is Standard & Premium SSD.

ODFC is enabled. 

The other questions I will try to find out. Will be back....

 
 

[#5]

And thank you for your reply! 🙂

[#6]

@EBRIN2233 

Analyze Resource Consumption:

PerfInsights: Consider using PerfInsights, a tool specifically designed for Azure VMs. It provides insights into Azure best practices, SQL best practices, I/O latency graphs, and detailed CPU and memory information. 

Existing Tools: Utilize familiar performance troubleshooting tools like Perfmon or Procmon (similar to on-premises servers). These tools can help identify resource-intensive processes and bottlenecks.

Common Factors for High-CPU Situations:

Recent Code Changes or Deployments: Check if recent code changes or deployments (especially for apps like Internet Information Services, Microsoft SharePoint, or Microsoft SQL Server) are contributing to the high CPU usage.

Updates: Investigate whether recent OS-level updates or application-level cumulative updates might be causing the issue.

Query Optimization: For SQL Server and Oracle data tier applications, outdated indexes or inefficient query plans can lead to increased CPU consumption.

Azure VM-Specific Processes: Certain processes (e.g., RDAgent, Monitoring Agent, MMA agent, or Security client) specific to Azure VMs may cause high CPU usage. 

Check VM Size and Storage:

Verify if the current VM size (D4s_v5) is sufficient for the workload. If both CPU usage and available memory are consistently high, consider upgrading to a larger VM size (e.g., D8s_v5) to better support your users’ needs.

Monitor and Optimize:

Continuously monitor resource utilization using Azure Monitor or other monitoring tools.

Optimize your application code, database queries, and indexing to reduce unnecessary resource consumption.

Review Extensions and Agents:

Ensure that any extensions or agents running on the VM (e.g., Monitoring Agent, MMA agent) are configured correctly and not causing excessive CPU load.

Remember that each environment is unique, so tailor your troubleshooting approach based on the specific characteristics of your client’s AVD setup. If the issue persists, consider reaching out to Azure support for further assistance.

[#7]

7. If you exclude FSLogix from the equation, is it still slow?? You can do this by adding users to the local FSLogix exclude group on the session hosts.

Excluding FSLogic -> same issues.

Disabeling OneDrive makes the environment fast. Enabling it makes the environment slow.

OneDrive and FSLogic on same storage but in different container.

Starting OneDrive on server as Admin (without shares and sync) makes the server slow also.

[#8]

so problems seem to pinpoint to OneDrive client / setup.

[#9]

Update:
We have installed a new VM from scratch, not using the default images provided by Microsoft.
We did the same with M365. After that we added the machine to the pool and deployed policies etc.

This machine seems to work fine - for now.
We will do some tests the upcoming days and I will keep you posted.

Our 'conlcusion' for now - something is not right in the default Win11 Pro image provided by MS in combination with settings/policies we have installed. 

[#10]

I have a customer that is experiencing slow performance regarding OneDrive/SharePoint when opening files on AVD Hosts.
The setup is:
- WVDadmin with a master VM that i make changed on and re-image.
- AVD is domain joined but NOT hybrid joined.
- 10-12 users on D4s V5 (4 vCPU and 16 GB RAM) AVD hosts. Started with two but tried with tre as well it gave no performance increase regarding OneDrive/SharePoint
- I have used MS own Windows 11 23H2 Multi session host without Office package.
- Installed Office package with shared computer activation WITHOUT teams and OneDrive included.
- Run https://github.com/The-Virtual-Desktop-Team/Virtual-Desktop-Optimization-Tool after image rollout, removed bunch og appx packages, services and scheduled tasks.
- Installed OneDrive as machine install (best prac for AVD) (newest version).
- FSLogix is the newest version on Azure Files Premium 490 GB Storage. Multi SMB Channel enabled.
- Implemented OneDrive cleanup by Marcel: https://blog.itprocloud.de/OneDrive-Clean-Up-For-Azure-Virtual-Desktop-AVD-agains-Profile-Blotting/
- Users use seperate dedicated mailbox that they need to roam credentials with and therefore roamidentity is enabled.

The Issue:
OneDrive/SharePoint files is like 20-30 seconds to open for the users. It makes no difference if the file is not on-demand or not. What is wrong here and what can be done so its faster for the users?

[#11]

Hi Oliver.
Our freshly installed VM's seemed to do the trick. For about three days.
We are experiencing the same issues like you have. Browsing through explorer on a "local' disk is fast. Browsing through explorer on SharePoint/OneDrive files is slow - 20-30 seconds per click.

We are working with a partner for a solution on this. But are considering a move to Azure File Storage instead of SharePoint - yes, the client will lose the M365 functionality, we know.

Do you have any updates?

[#12]

@EBRIN2233 HI,

It worked to deploy a fresh AVD host without using the VDOT or Marcels OneDrive cleanup. Im seeing pretty normal performance for the users. The only problem is i have nothing to dehydrate the files for the users. 

Disk is 128 GB SSD Premium and IOPS is bumped up to 1100. Im running 1:1 ratio with the vCPU per user.

[#13]

I was the only user just now on a D4S-v5 so I had 4vCPU and 16GB to play with.
Everything is fast and great to work with, only files and folders in SharePoint/OneDrive are an issue.

We now disabled to connect local drives in the AVD-session. One of my collegues read that could cause issues like this.

But my guess it's a OneDrive/SharePoint issue somehow.

We disabled the GPO to connect the sharepoint libraries automatically. So the GPO can't be the cause either.

[#14]

The performance when browsing through OneDrive/SharePoint files is still super slow. I Can see when i try to open a file onedrive create a extra task in jobmanager. Hope the OneDrive team fixes this soon! Way to slow for the users 😞

[#15]

@Oliver_Krage - same here. But we hear several of our customers complain about this the last 3-4 days - on local devices!

At first we did not match our AVD problems with the problems on local devices but we do have a premonition that the OneDrive client could be the cause of all these issues. 

Keep you posted!

[#16]

I do not see any issues locally regarding OneDrive. Only on AVD hosts 😞

[#17]

We have been working on this issue for months with Microsoft support, but still no solution. Very frustrating and very poor support.

[#18]

I found the solution at least in our enviroment. The customer needed Citrix Workspace installed on the AVD host (dont ask why). This slows the entire file explorer + OneDrive down when its running in the background. If i close Citrix Workspace for my test user, it went pretty fast.

[#19]

HI Marantz. That's frustrating...
We see issues on AVD and locally with OneDrive. So the focus of MS Support team is on OneDrive at the moment.

[#20]

I've read similar stories about Adobe Reader on AVD. But was not taking that to seriously....
Perhaps I have te reconsider now... after reading your post.
Thanks Oliver!