Forum Discussion
Teams site showing up in public Bing results
It doesn't show up in Google, but it does show up in Yahoo and DuckDuckGo which if I am correct, use the Bing engine. See screenshot below. I have blocked out the search terms as this is an ongoing project.
- David Rosenthal
Microsoft
I have a site level repro of this. Not down to documents or folders, but the site itself. This is a modern team site spun up as the result of creation in Microsoft Teams, which fired the corresponding Office 365 Group and everything that comes with that.
Something seems to be visible to the search engines around URLs at least. VesaJuvonen, AdamHarmetz this is probably something you guys would want to know about.
- VesaJuvonen
Thx David Rosenthal for looping us on this. Started internal investigation right away to avoid confusion.
- AdamHarmetz
Thanks for looping us in.
If a search crawler discovers a link to an authenticated SharePoint Online site, it may add the link to the index. Because the site requires authentication, the site title and contents will not be indexed – only the presence of the URL. This should only occur when there is a link to the site collection somewhere on the public internet (e.g. someone might have use anonymous link sharing and posted that link somewhere where a internet search crawler could find it).
The timing of this particular thread is an interesting coincidence, as we are doing work in this area. Just this month, to mitigate this, we have added a default robots.txt file to every site collection which instructs search crawlers not to index this URL. This will prevent search crawlers from adding new sites to their index. Existing sites should be removed from the search results next time the site is indexed by the search crawler.
You can actually see the robots.txt file for your domain at https://<tenantURL>/robots.txt The change is rolling out - should be mostly complete but might be a few deployments that do not yet have it.
Hope this helps!
