开发者

can i use robots.txt while handling my site with htaccess

开发者 https://www.devze.com 2022-12-13 17:17 出处:网络
I am using htaccess in my site, such that all the request to my site will be redirected to index page in开发者_运维百科 my root directory. No other file in my site can be accessed because my htaccess

I am using htaccess in my site, such that all the request to my site will be redirected to index page in开发者_运维百科 my root directory. No other file in my site can be accessed because my htaccess will restrict it. My doubt is, when I use robots.txt file, will the search engines be able to reach the robots.txt file in my domain?. Or must i modify my htaccess file to allow the search engines to read the robots.txt file. If so help me in finding that specific code for htaccess.


I suppose you're using some sort of rewriting. You can exclude files from beeing processed by mod_rewrite with the following rule:

RewriteCond %{REQUEST_FILENAME} -f
RewriteRule ^(.*) $1 [L]

In this case all files that do exist in the webroot and are called directly will not be processed any further. Replace _%{REQUEST_FILENAME} with robots.txt and you should be fine.

Best wishes,
Fabian


How about trying if you can yourself access the robots.txt via a web browser? If you can, then the search engines can, an vice versa (unless you're using some IP- or browser-specific redirections or such). That is, http://yoursite.com/robots.txt

0

精彩评论

暂无评论...
验证码 换一张
取 消