How can I prevent search engines indexing pages with 3 or more forw开发者_运维百科arslashes in their paths? for example:
www.example.com/about.html ->ok
www.example.com/1/2/3/4/5/test.html -> no index!!!
How should I write the robots.txt? Thanks!
Try with
User-Agent: *
Disallow: /*/*/*/*
Although I'm not sure all bots support the *.
加载中,请稍侯......
精彩评论