Why the Lithium Platform is setup to block content crawling by Google, Bing, etc ... ?

%3CLINGO-SUB%20id%3D%22lingo-sub-13210%22%20slang%3D%22en-US%22%3EWhy%20the%20Lithium%20Platform%20is%20setup%20to%20block%20content%20crawling%20by%20Google%2C%20Bing%2C%20etc%20...%20%3F%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-13210%22%20slang%3D%22en-US%22%3E%3CP%3EWe%20were%20kickout%20Yammer%20%3D%26gt%3B%20Lithium%20for%20different%20reasons%20(advantage%20%3F)%20and%20make%20community%20conversation%20more%20accessible%2Fvisible%20to%20the%20rest%20of%20the%20world.%26nbsp%3BMy%20question%20is%20the%20following%2C%20if%20I%20correctly%20understood%20the%20objective%20%3CSTRONG%3E(WHY)%20the%20Lithium%20platform%20is%20setup%20to%20disallow%20content%20indexing%20from%20Search%20Engine%20(and%20btw%20not%20allow%20post%20and%20conversation%20made%20within%20the%20platform%20searchable%20over%20Internet)%20%3F%3C%2FSTRONG%3E%3C%2FP%3E%3CP%3E%3CSTRONG%3E%26nbsp%3B%3C%2FSTRONG%3E%3C%2FP%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Frobots.txt%22%20target%3D%22_blank%22%3Ehttps%3A%2F%2Ftechcommunity.microsoft.com%2Frobots.txt%3C%2FA%3E%3C%2FP%3E%3CPRE%3EUser-agent%3A%20*%0ACrawl-delay%3A%205%0ADisallow%3A%20%2F%3C%2FPRE%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EThanks%20for%20feedback%20%3A)%3C%2Fimg%3E%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EEric%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-13956%22%20slang%3D%22en-US%22%3ERe%3A%20Why%20the%20Lithium%20Platform%20is%20setup%20to%20block%20content%20crawling%20by%20Google%2C%20Bing%2C%20etc%20...%20%3F%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-13956%22%20slang%3D%22en-US%22%3E%3CP%3EThe%20content%20of%20the%20network%20is%20now%20indexed%2C%20the%20robots.txt%20was%20updated%20by%20Microsoft%20Team%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CPRE%3EUser-agent%3A%20*%0ACrawl-delay%3A%205%0ADisallow%3A%20%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F(*)%0ADisallow%3A%20%2Ft5%2Fforums%2Ffilteredbylabelpage%2Fboard-id%2F(*)%0ADisallow%3A%20%2Ft5%2Ftag%2FQuery%2Ftg-p%2Fboard-id%2F(*)%0ADisallow%3A%20%2Ft5%2Fforums%2Fpostpage%0ADisallow%3A%20%2Ft5%2Fforums%2Freplypage%0ADisallow%3A%20%2Ft5%2Fforums%2Ftagdetailpage*%0ADisallow%3A%20%2Ft5%2Fforums%2Ftagleaderboardpage*%0ADisallow%3A%20%2Ft5%2Fforums%2Fsearchpage%0ADisallow%3A%20%2Ft5%2Fauthentication*%0ADisallow%3A%20%2Ft5%2Fmedia*%0ADisallow%3A%20%2Ft5%2Fforums%2Fforumtopicprintpage%0ADisallow%3A%20%2Ft5%2Fideas%2Fideaprintpage%0ADisallow%3A%20%2Ft5%2Fblogs%2Fblogarticleprintpage%0ADisallow%3A%20%2Ft5%2Fhelp%2Ffaqpage%0ADisallow%3A%20%2Fgxcuf89792%2Frss%0ADisallow%3A%20%2Ft5%2Fforums%2Fusersonlinepage%0ADisallow%3A%20%2Ft5%2Fforums%2Frecentpostspage%0A%0ADisallow%3A%20%2F*%3Ft%3Aac%3D%0ADisallow%3A%20%2F*%26amp%3Bt%3Aac%3D%0ADisallow%3A%20%2Ft5%2Futil%2Fcomponentrenderpage%0ADisallow%3A%20%2F*%3Alightboxrendercomponent%3F%0ADisallow%3A%20%2F*%3Alightboxrendercomponent%26amp%3B%0ADisallow%3A%20%2F*%3Alightboxrendercomponent%2F%0ADisallow%3A%20%2F*%3Fticket%3D%0ADisallow%3A%20%2F*%26amp%3Bticket%3D%0A%0ADisallow%3A%20%2Ft5%2Ftkb%2Farticlehistorypage%0ADisallow%3A%20%2Ft5%2Fforums%2Freplypage%3C%2FPRE%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-13290%22%20slang%3D%22en-US%22%3ERe%3A%20Why%20the%20Lithium%20Platform%20is%20setup%20to%20block%20content%20crawling%20by%20Google%2C%20Bing%2C%20etc%20...%20%3F%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-13290%22%20slang%3D%22en-US%22%3E%3CP%3EThanks%20for%20feedback%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F521%22%20target%3D%22_blank%22%3E%40Sergei%20Baklan%3C%2FA%3E%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-13249%22%20slang%3D%22en-US%22%3ERe%3A%20Why%20the%20Lithium%20Platform%20is%20setup%20to%20block%20content%20crawling%20by%20Google%2C%20Bing%2C%20etc%20...%20%3F%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-13249%22%20slang%3D%22en-US%22%3E%3CP%3EI%20guess%20it%20will%20be%20changed%20in%20coming%20update%2C%20see%20%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2FCommunity-Announcements%2FSept-16-Release-Fixes-and-Enhancements-to-the-Microsoft-Tech%2Fm-p%2F11850%23M63%22%20target%3D%22_self%22%3Ehttps%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2FCommunity-Announcements%2FSept-16-Release-Fixes-and-Enhancements-to-the-Microsoft-Tech%2Fm-p%2F11850%23M63%3C%2FA%3E%3C%2FP%3E%3C%2FLINGO-BODY%3E
Contributor

We were kickout Yammer => Lithium for different reasons (advantage ?) and make community conversation more accessible/visible to the rest of the world. My question is the following, if I correctly understood the objective (WHY) the Lithium platform is setup to disallow content indexing from Search Engine (and btw not allow post and conversation made within the platform searchable over Internet) ?

 

https://techcommunity.microsoft.com/robots.txt

User-agent: *
Crawl-delay: 5
Disallow: /

 

Thanks for feedback :)

 

Eric

3 Replies

The content of the network is now indexed, the robots.txt was updated by Microsoft Team

 

User-agent: *
Crawl-delay: 5
Disallow: /t5/user/viewprofilepage/user-id/(*)
Disallow: /t5/forums/filteredbylabelpage/board-id/(*)
Disallow: /t5/tag/Query/tg-p/board-id/(*)
Disallow: /t5/forums/postpage
Disallow: /t5/forums/replypage
Disallow: /t5/forums/tagdetailpage*
Disallow: /t5/forums/tagleaderboardpage*
Disallow: /t5/forums/searchpage
Disallow: /t5/authentication*
Disallow: /t5/media*
Disallow: /t5/forums/forumtopicprintpage
Disallow: /t5/ideas/ideaprintpage
Disallow: /t5/blogs/blogarticleprintpage
Disallow: /t5/help/faqpage
Disallow: /gxcuf89792/rss
Disallow: /t5/forums/usersonlinepage
Disallow: /t5/forums/recentpostspage

Disallow: /*?t:ac=
Disallow: /*&t:ac=
Disallow: /t5/util/componentrenderpage
Disallow: /*:lightboxrendercomponent?
Disallow: /*:lightboxrendercomponent&
Disallow: /*:lightboxrendercomponent/
Disallow: /*?ticket=
Disallow: /*&ticket=

Disallow: /t5/tkb/articlehistorypage
Disallow: /t5/forums/replypage