Forum Discussion
Robots.txt tester failed
Hi, I would like to understand the reason why my robots.txt tester keeps on failing. I have emailed tech support who told me to review my sitemap xml file, however I’m running 3 blogs with identica...
Thank you for your reply, ctomczyk . Will defo try that.
An additional detail though, my blog is updated several times a week and the newer posts are getting indexed. However the old ones are not, I even checked on the url inspection where the status stated “Discovered but not crawled”.
Even after I fetched the sitemap or submit the urls, the discovery date never changed nor the status move on to crawling stage.
Do you happen to have any suggestions for this?
shenyihui “Discovered but not crawled” - that might be the issue with canonical link or you may have duplicated content so the search engines sees same content under the two different URLs.
> Even after I fetched the sitemap or submit the urls, the discovery date never changed nor the status move on to crawling stage.
If the sitemap contains correct URLs then all you can do is wait until search engine will update their data. Or eventually contact Webmaster Support - Bing Webmaster Tools: https://www.bing.com/webmasters/help/webmasters-support-24ab5ebf
> Even after I fetched the sitemap or submit the urls, the discovery date never changed nor the status move on to crawling stage.
If the sitemap contains correct URLs then all you can do is wait until search engine will update their data. Or eventually contact Webmaster Support - Bing Webmaster Tools: https://www.bing.com/webmasters/help/webmasters-support-24ab5ebf
