SharePoint Search

[Pieter Veenstra]

Today my customer reported an issue with search in SharePoint Online. They have a search box web part, a search result web part and a refinement panel:

RefinableString00 is mapped to a Client Code field.

Within the Refinement panel I'm seeing 16 results for client code "XYZ123". This is the expected result.

When I search for  XYZ123 I'm getting just 1 result returned ( I checked the show duplications setting in the web part and that is fine)

Then I typed in my search box the following:

"RefinableString00:XYZ123"

and now I'm getting 16 results returned. Any ideas why the search query without RefinableString00: returns 1 result where the query with RefinableString00: returns 16. Wouldn't the query without RefinableString00: search through every column available for searching?

[#2]

It looks like something is not going ok with the search but what good question have you tried to search on the crawled property?

[#3]

RefinableString00 is my Managed property that is mapped to my crawled property. I don't think that it is possible to search directly on the Crawled property name or am I missing something?

[#4]

Hi @Deleted,

I might have found the problem. 

I just forced a recrawl of one of the lists ( list settings -> advanced settings -> reindex list ) and now data from that list is appearing. 

So I'm guessing that the following has happened in this order:

1. The many lists containing the data were created.

2. The data was added.

3. The Managed Property RefinableString00 was mapped.

4. The only list that was showing data was created

5. The continuous crawl picked up the data form the new list.

I will now have a look at forcing the reindexing on the other lists as well.

I still don't understand however why specifically searching on RefinableString00 does work and General searching does not work

[#5 · Accepted solution]

Just to complete this issue. I've created a PowerShell script that touches all the list items in my site collections, so that search will pick up these items. Problem resolved for me! Not sure how this customer got into this situation though. 

The script can be found below: 

$cred = Get-Credential -Message "Password" -UserName admin@mytenant.onmicrosoft.com
Connect-PnPOnline -Url https://mytenant-admin.sharepoint.com -Credentials $cred
$sites = Get-PnPTenantSite | where { $_.Url -like "https://mytenant.sharepoint.com/teams*"}
foreach ( $site in $sites)
{
Connect-PnPOnline -Url $site.Url -Credentials $cred
$web = Get-PnPWeb

$list = Get-PnPList -Web $web | Where { $_.Title -eq "MyList" }
$listItem = Get-PnPListItem -List $list
$listItem.Update();
$web.Context.Load($listItem);
$web.Context.ExecuteQuery();
$listItem

}

[#6]

Why do you not use the script from @Mikael Svenson. to reindex tenant to be sure.

param([Parameter(Mandatory=$true,ValueFromPipeline=$true)]$url, [Parameter(ValueFromPipeline=$true)][string]$username, [Parameter(ValueFromPipeline=$true)][string]$password, [ValidateSet('skip','on','off')][System.String]$enableAllManagedProperties="skip"  )
# Re-index SPO tenant script, and enable ManagedProperties managed property
# Author: Mikael Svenson - @mikaelsvenson
# Blog: http://techmikael.blogspot.com

# Modified by Eric Skaggs on 10/21/2014 - had trouble running this script as it was; functionality has not been changed

function Reset-Webs( $siteUrl ) 
{
	$clientContext = New-Object Microsoft.SharePoint.Client.ClientContext($siteUrl)
	$clientContext.Credentials = $credentials 
	 
	if (!$clientContext.ServerObjectIsNull.Value) 
	{ 
		Write-Host "Connected to SharePoint Online site: '$siteUrl'" -ForegroundColor Green 
	} 

	$rootWeb = $clientContext.Web
	processWeb($rootWeb)	
}

function processWeb($web)
{
	$subWebs = $web.Webs
	$clientContext.Load($web)		
	$clientContext.Load($web.AllProperties)
	$clientContext.Load($subWebs)
	$clientContext.ExecuteQuery()
	
	Write-Host "Web URL:" $web.Url -ForegroundColor White
	if( $enableAllManagedProperties -ne "skip" ) {
		Set-AllManagedProperties -web $web -clientContext $clientContext -enableAllManagedProps $enableAllManagedProperties
	}
	
	[int]$version = 0
	$allProperties = $web.AllProperties
	if( $allProperties.FieldValues.ContainsKey("vti_searchversion") -eq $true ) {
		$version = $allProperties["vti_searchversion"]
	}
	$version++
	$allProperties["vti_searchversion"] = $version
	Write-Host "-- Updated search version: " $version -ForegroundColor Green
	$web.Update()
	$clientContext.ExecuteQuery()
	
	# No need to process subwebs if we only mark site collection for indexing
	if($enableAllManagedProperties -ne "skip") {
		foreach ($subWeb in $subWebs)
		{
			processWeb($subWeb)
		}
	}
}

function Set-AllManagedProperties( $web, $clientContext, $enableAllManagedProps )
{
	$lists = $web.Lists
	$clientContext.Load($lists)
    $clientContext.ExecuteQuery()
 
    foreach ($list in $lists)
    {
        Write-Host "--" $list.Title
		
		if( $list.NoCrawl ) {
			Write-Host "--  Skipping list due to not being crawled" -ForegroundColor Yellow
			continue
		}
 
		$skip = $false;
		$eventReceivers = $list.EventReceivers
		$clientContext.Load($eventReceivers)
		$clientContext.ExecuteQuery()
		
		foreach( $eventReceiver in $eventReceivers )
		{
			if( $eventReceiver.ReceiverClass -eq "Microsoft.SharePoint.Publishing.CatalogEventReceiver" ) 
			{
				$skip = $true
				Write-Host "--  Skipping list as it's published as a catalog" -ForegroundColor Yellow
				break
			}
		}
		if( $skip ) {continue}
 
		$folder = $list.RootFolder
		$props = $folder.Properties
		$clientContext.Load($folder)		
		$clientContext.Load($props)
		$clientContext.ExecuteQuery()
		
		if( $enableAllManagedProps -eq "on" ) {
			Write-Host "--  Enabling all managed properties" -ForegroundColor Green
			$props["vti_indexedpropertykeys"] = "UAB1AGIAbABpAHMAaABpAG4AZwBDAGEAdABhAGwAbwBnAFMAZQB0AHQAaQBuAGcAcwA=|SQBzAFAAdQBiAGwAaQBzAGgAaQBuAGcAQwBhAHQAYQBsAG8AZwA=|"
			$props["IsPublishingCatalog"] = "True"
		}
		if( $enableAllManagedProps -eq "off" ) {
			Write-Host "--  Disabling all managed properties" -ForegroundColor Green
			$props["vti_indexedpropertykeys"] = $null
			$props["IsPublishingCatalog"] = $null
		}
		$folder.Update()
		$clientContext.ExecuteQuery()		
    }
}

Import-Module Microsoft.Online.SharePoint.PowerShell
# change to the path of your CSOM dlls and add their types
$csomPath = "c:\Program Files\Common Files\microsoft shared\Web Server Extensions\15\ISAPI"
Add-Type -Path "$csomPath\Microsoft.SharePoint.Client.dll" 
Add-Type -Path "$csomPath\Microsoft.SharePoint.Client.Runtime.dll" 

if([String]::IsNullOrWhiteSpace($username)) {
	$username = Read-host "What's your username?"
}

if([String]::IsNullOrWhiteSpace($password)) {
	$securePassword = Read-host "What's your password?" -AsSecureString 
} else {
	$securePassword = ConvertTo-SecureString $password -AsPlainText -Force 
}

$credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($username, $securePassword) 
$spoCredentials = New-Object System.Management.Automation.PSCredential($username, $securePassword)
Connect-SPOService -Url $url -Credential $spoCredentials
Get-SPOSite | foreach {Reset-Webs -siteUrl $_.Url }

[#7]

Remember to add -Limit ALL to the Get-SPOSite, as I committed yesterday to my repo https://github.com/wobba/SPO-Trigger-Reindex/blob/master/reindex-tenant.ps1 :)

[#8]

This is not necessarily weird. If you have a regular list column, and map the crawled property for this column to RefinableString00, you effectively remove the content in that column as being full-text searchable. RefinableString00 is not a searchable managed property.

If it's a site column, you should get the same results as the ows_q_col crawled property is searchable, while you map the ows_col and make it non searchable.

I have a table at http://www.techmikael.com/2014/07/what-makes-sharepoint-column-searchable.html which shows how this works.

[#9]

HI @Mikael Svenson,

that is what I originally thought too as I inspected the RefinableString00 property. But Then I wouldn't have expected the "RefinableString00:XYZ123" to give the correct result either. Unless of course searching for "RefinableString00:XYZ123" isn't considered to be searching.

also the crawled property is marked as Include in full-text index

[#10]

Correct, mp:something is a property query, not full-text search. Full-text(free text) search is the searchable property, while a property query is the queryable property of a managed property.

[#11]

I have a very minor naive query. In my SharePoint Online library, I have a Power Point document in which it has a text box with value "SCOPE-068-T-CMS101-2017-v1.0". If I do a SharePoint search with "full value" then the search correctly shows the document in search list. But if I search with a partial value say "SCOPE-068-T-CMS101-2017" then it doesn't show the document in search results.

Is there some search setting which can control this behavior?  Thanks for the help. 

[#12]

Hey 

Mikael,

I am having a heck of a time retrieving search results (i.e., data from lists) from my communication site landing page.  If I were to go to site contents, and plop something into the search box, then all results come back.  My gut is telling me this may have something to do with managed/crawled properties.  Finally, the drop-down selections (this site, everything, people, etc) do not appear in the search box on the landing/Home page as well.  They do, however, appear when accessing a library, site contents, etc.  I have admin privilieges, but have not been able to solve the problem.  Any ideas?

Thanks much,

Paul

[#13]

@Paul Martello was there a question in there? :) As for scoped search in the search box, this is not possible and there is a user-voice out there to allow us to pick what that box searches in.

My gut feel is that the magic behind the modern search box puts some filters on what is actually returned, where as the classic box does not.

[#14]

Hi Vivek,
It seems that this is related to word breaking or Tokenization capabilities of SharePoint search. If I test with the same text as you have, I see that with the partial query and using wildchar, the highlighted hit on the match is always before the dot.

There were tokenization properties announced last year for SharePoint Online last year.

See: https://blogs.technet.microsoft.com/beyondsharepoint/2017/06/17/search-extended-word-breaking/

However these are only available for custom managed properties. In this case, what you didn't mention is whether you use full text search or a custom managed property. I hope the link here can point you to the right direction.

[#15]

Thanks @Mikael Svenson !  Your responses are helpful and clarify a complex topic.