Forum Discussion

Eric JENOUVRIER's avatar
Eric JENOUVRIER
Iron Contributor
Sep 15, 2016

Why the Lithium Platform is setup to block content crawling by Google, Bing, etc ... ?

We were kickout Yammer => Lithium for different reasons (advantage ?) and make community conversation more accessible/visible to the rest of the world. My question is the following, if I correctly understood the objective (WHY) the Lithium platform is setup to disallow content indexing from Search Engine (and btw not allow post and conversation made within the platform searchable over Internet) ?

 

https://techcommunity.microsoft.com/robots.txt

User-agent: *
Crawl-delay: 5
Disallow: /

 

Thanks for feedback :)

 

Eric

      • Eric JENOUVRIER's avatar
        Eric JENOUVRIER
        Iron Contributor

        The content of the network is now indexed, the robots.txt was updated by Microsoft Team

         

        User-agent: *
        Crawl-delay: 5
        Disallow: /t5/user/viewprofilepage/user-id/(*)
        Disallow: /t5/forums/filteredbylabelpage/board-id/(*)
        Disallow: /t5/tag/Query/tg-p/board-id/(*)
        Disallow: /t5/forums/postpage
        Disallow: /t5/forums/replypage
        Disallow: /t5/forums/tagdetailpage*
        Disallow: /t5/forums/tagleaderboardpage*
        Disallow: /t5/forums/searchpage
        Disallow: /t5/authentication*
        Disallow: /t5/media*
        Disallow: /t5/forums/forumtopicprintpage
        Disallow: /t5/ideas/ideaprintpage
        Disallow: /t5/blogs/blogarticleprintpage
        Disallow: /t5/help/faqpage
        Disallow: /gxcuf89792/rss
        Disallow: /t5/forums/usersonlinepage
        Disallow: /t5/forums/recentpostspage
        
        Disallow: /*?t:ac=
        Disallow: /*&t:ac=
        Disallow: /t5/util/componentrenderpage
        Disallow: /*:lightboxrendercomponent?
        Disallow: /*:lightboxrendercomponent&
        Disallow: /*:lightboxrendercomponent/
        Disallow: /*?ticket=
        Disallow: /*&ticket=
        
        Disallow: /t5/tkb/articlehistorypage
        Disallow: /t5/forums/replypage

Resources