Forum Discussion
Cloud Hybrid Search Bandwidth and Throttling
We are testing the waters with cloud hybrid search and I currently have it indexing our on-prem SharePoint server of around 5M items and 4TB of content. Our normal index is 12M items across the intr...
I'm not aware of any guidance but I do cover this in conference sessions I give on hybrid search. I have a bandwidth model (eg a spreadsheet) I made around this.
You can use QoS and/or Crawler Impact rules to slow things down. I definitely suggest providing some bandwidth limiter (like seperate VLANs or a QoS policy), because crawling will take all the bandwidth you give it. 200 docs/s is about what I'd expect if you are crawling a file system.
1 TB/week is only ~1.6 MB/s average, so presumably your crawls are completing and your link is not busy all the time. You are only transmitting indexable text so depending on the content mix this could be 10 TB of content CHANGING every week.....if that does not seem like it matches what's happening in your environment I would check to be sure you are doing incrementals and don't have full crawls scheduled all the time or something like that.