hybrid federated search from SharePoint Server to SharePoint Online inconsistent search result

[Martin_Ericsson]

Hi,

I have configured hybrid federated search from SharePoint server to SPO. It works but we are experiencing a strange behavior. When we do a search from the search center (SP on-prem) from time to time we need to press search several times in order to get a search result from SPO.

I have traced it down in the ULS and see the following entry every time when the search fail to get a result from SPO:

NodeRunner.exe (0x1C08)    0x24F4    SharePoint Server Search    Query    aqdw1    Medium    QueryRouterEvaluator: Received results for already timed out query with query type ResultBlock    543aaf9c-0a9c-481a-9cfa-dda4fbcc4e5a

We use SP2019 with latest CU. We have this problem in two different environments on 2 different o365 tenants and data centers for SP on-prem. I have attached a trimmed ULS log. Where we can see the problem occur 2 times and by the 3 time the same search query is done, it works and SPO result is shown correctly.

Anyone experienced this problem?

Rgds Martin

[#2]

@Martin_Ericsson Did you resolve this issue? I am trying to set up outbound hybrid federated search and I am getting this same timeout error in the ULS logs. In my case, I am trying to display the SP online search results in a result block and the majority of time they are not displayed at all and only the on-prem results are displayed. Were you able to track down the root cause?

[#3]

In this procedure, you create a result source in the SharePoint Server deployment. This result source is a definition that specifies SharePoint Online as a provider to get search results from. This definition specifies each of the following:

The SharePoint Online URL to get search results from

The protocol for getting those results

The method for authenticating against SharePoint Online

Result sources can be created at the Search service application level, the site collection level, or the site level. In this procedure, you create the result source at the Search service application level. This will make the result source available to any query rule that is created at the same level, and also to any query rule that is created for a site collection or site that is in a web application that consumes the Search service application.

[#4]

@Lewis-H In my case, the result source and the query rule seem to be configured correctly as it intermittently works properly. Also, if I do the opposite where the primary results are the online results and the result block displays the on-prem results then it works fine.

There are a lot of components involved here and most of them are not easily accessible (on-prem search components, SP online search, etc.) to view exactly what is happening. I was interested in knowing what causes may have been found and what they used for troubleshooting.

[#5]

@Rob_DF Sorry for the late answer. No we didn't resolve this. We choose a different path and stopped using the Hybrid Federated Search since it was so unstable and unreliable. Were you able to resolve it?

[#6]

@Martin_Ericsson 

I finished configuring Outbound (& Inbound) hybrid Federated search about a month ago.
This seems like an expected behavior where only On-premises results are shown more often than not.

This indeed is a time out issue where Onprem result come first and it stops looking for SPO results in block.

I am planning to open an Advisory case with Microsoft. Will post here if I have any updates on that.