Copilot searching SharePoint inconsistent

I don't think you can, it seems to be a fundamental limitation of CoPilot in SharePoint that makes it virtually useless and is due to the way it uses the data it has access to.

Example. Create a new SharePoint site. In a document library upload 100 simple consultant profiles. Consultants are either Developers, Project Managers, Business Analysts or Scrum Masters. Each profile contains some technical proficiencies and work history. Each profile is less than 1000 characters and all profiles are formatted identically.

Now ask CoPilot some simple questions about this data. Half the time it won't even correctly tell you how many profiles are in the document library. Ask it how many Developer profiles or Business Analyst profiles there are and it won't have a clue. It will give you answers but they will all be incorrect. Ask it how many consultants have worked for a particular client or in a particular industry sector and the answers will all be nonsense.

Keep in mind that the total amount of data used in this example is less than a 1MB and CoPilot cannot process it, the SharePoint tenant of an average sized enterprise will contain orders of magnitude more data than this, and CoPilot will perform even worse.

This is because of how CoPilot works. It doesn't actually process all the data it has access to, so it can't usefully aggregate anything. If you ask it how much holiday entitlement you have it will basically look for files containing "holiday entitlement" in the title or content, pick several of those files, and see whether the answer happens to be in the files it picked. If the answer isn't in those files it will either say it can't answer the question or it will invent a nonsense answer.

What's more it will only search documents in the site in which you ask it the question, so using CoPilot on a large corporate SharePoint intranet contianing many sites you have to already know which site contains the information you need.

This is actually worse than the default search in SharePoint. Assuming you have your site designed so that search rolls up to the Home site you can perform a search for "holiday entitlement" at the root level of your intranet and SharePoint will identify any content with that name or containing that term anywhere in the tenant. Then you just open the Holiday Entitlement policy document and read or search that document.

The sad truth is that currently CoPilot in SharePoint isn't even as useful as SharePoint's standard search (which itself isn't very good), and as for CoPilot giving you instant access to all your business data in a ready to use manner that will revolutionise your business, I'm afraid MS is not being honest.

Maybe in a few years it will be able to do this, but right now MS is charging a fortune in extra license costs for something that doesn't really deliver any extra value at all.

Thank you for sharing the very poor integration with CoPilot and Sharepoint. I have struggled with this exact thing.

Might be possible;
1. Not all files might be indexed for search (issue with search indexing service)

2. The search might be limited to certain libraries or sites within SharePoint. If the search scope is restricted, won’t see results from outside that scope.